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1  Summary 

The  MICA  Open  Experimentation  Platform  (OEP)  successfully  provided  an  integration 
framework  for  MICA  controllers  and  experimentation  platform  for  simulation, 
demonstration,  assessment  and  transition  of  MICA  research  to  operational  systems. 
Accompanying  Challenge  Problems  (CP)  provided  a  challenging  environment  and  C2 
products  representative  of  real  world  Intelligence  Preparation  of  the  Battlefield  (IPB), 
Commander’s  Guidance  and  Rules  of  Engagement  (ROE).  The  CPs  also  incorporated  a 
red  force  controller  that  enabled  intelligent  adversarial  reactions  including  an  Integrated 
Air  Defense  System  (IADS).  The  combined  OEP/CP  capabilities  represented  a 
challenging  but  manageable  3  to  5  day  air  campaign  encompassing  all  elements  of  the 
Target  Kill  Chain.  The  OEP  and  CP  development  strategy  relied  on  early  release  of 
initial  capabilities  with  frequent  incremental  upgrades  prioritized  to  meet  multiple 
research  objectives.  This  strategy  was  successfully  employed  with  the  initial  release  of  a 
substantial  capability  occurring  within  60  days  after  contract  award.  This  early  release 
was  crucial  to  enticing  researchers  to  use  the  OEP  rather  than  their  in-house  simulations. 
Subsequent  OEP  and  CP  releases  consistently  led  researcher  needs  such  that  critical 
research  was  never  hindered  by  a  lack  of  OEP  capability. 

Boeing  planned  to  conduct  extensive  experimentation  as  described  in  Appendix  A. 
Unfortunately,  MICA  research  was  pre-maturely  terminated  due  to  evolving  priorities 
within  DARPA.  Only  two  of  the  available  controllers  were  submitted  to  Boeing  for 
evaluation  and  both  were  less  capable  than  would  have  been  the  case  if  planned  research 
had  been  completed.  Never  the  less  certain  conclusions  can  be  drawn  from  the  limited 
research.  Both  controllers  exhibited  a  hierarchical  decomposition  of  the  challenge 
problem  and  allocated  teams  of  assets  to  subsets  of  known  targets.  However,  the 
construct  of  their  team  appears  more  to  be  a  product  of  the  logical  partitioning  of  the 
challenge  problem  for  planning  simplification,  rather  than  the  grouping  of  assets  that 
dynamically  and  purposefully  cooperate.  Both  controllers  provide  capability  to  address 
the  inherent  uncertainty  of  knowledge  of  the  battlespace  such  as  Draper's  Information 
Model  and  Iterativity's  VII  IPB  interface.  Both  controllers  have  a  broad  Variable 
Initiative  Interface  (VII),  albeit  of  varying  levels  of  maturity  and  utility.  Both  controllers 
use  Commander’s  Guidance  and  ROE  to  adjust  target  values  but  do  not  actively  drive 
team  composition,  tasking  or  tactics  based  on  guidance  or  ROE.  Both  controllers 
evidenced  coordinated  team  tactics  and  planned  asset  flight  paths  in  response  to  either 
team  assignment  or  task  allocation.  Both  rely  on  a  centralized  planning  and  control 
paradigm  in  which  all  replans  are  global  and  are  executed  when  changes  to  the 
centralized  database  occur.  Unfortunately,  some  of  these  replans  were  unnecessary 
(insignificant  change  to  the  centralized  database)  or  should  have  been  local  rather  than 
global.  Furthermore,  since  each  controller  reacts  to  a  limited  subset  of  available  events, 
many  needed  replans  do  not  occur.  For  example,  neither  controller  reacts  to  threat 
warning  receiver  events.  The  result  is  that  all  intra-team  coordination  is  the  consequence 
of  a  priori  centralized  scheduling  or  sequencing.  There  is  no  provision  for  rapid 
localized  coordinated  activities  involving  individual  vehicles  or  groups  of  vehicles. 
Details  of  our  experimentation  results  are  provided  in  section  5. 
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The  shortcomings  of  these  controllers  can  best  be  appreciated  through  comparison  with 
traditional  air  campaign  C2  approaches.  In  September  2003  MICA  personnel  visited  to 
133rd  Iowa  Air  National  Guard  (ANG)  to  observe  how  they  would  prosecute  the 
Challenge  Problem.  The  intent  of  the  visit  was  to  develop  a  baseline  to  which  MICA 
controllers  could  be  compared.  The  133rd  ANG  took  a  completely  top-down  approach  to 
the  problem.  They  allocated  available  resources  into  five  primary  teams,  each  composed 
to  address  their  assigned  objectives.  The  two  largest  teams  were  tasked  to  eliminate  the 
enemy  IADS  by  destroying  their  C2  facilities.  Two  smaller  teams  were  tasked  with 
detecting  and  neutralizing  enemy  SSMs.  The  final  team  was  held  in  reserve  to  protect  the 
blue  base,  if  necessary. 

The  133rd  ANG  approach  to  problem  decomposition  was  "solution  centric"  or  effects 
based,  i.e.  "to  defeat  the  enemy,  we  must  do  this,  and  this,  and  this...”  and  their  solution 
started  with  a  functional  decomposition.  In  their  effects-based  approach  they:  1) 
developed  a  hypothesis  of  the  battle  space  based  on  available  IPB,  2)  selected  specific 
actions  to  elicit  responses  from  the  uncertain  enemy  force  structure  to  improve  situational 
awareness,  3)  chose  major  "high  value"  objective  i.e.  disable  C2  facilities,  4)  established 
an  order  to  the  battle,  5)  task  two  separate  simultaneous  "waves"  at  C2  facilities,  and  5) 
established  a  multi-pronged  plan  of  attack  that  avoided  sequential  flight  through  layers 
Air  Defenses  to  reach  the  chosen  objective.  The  ensuing  functional  decomposition 
focused  on  team  composition  and  task  assignment.  Two  teams  were  composed  to  strike 
against  the  enemy  center  of  gravity  ~  C2  facilities.  Wave  one,  tasked  to  find  and  locate 
SAMs  included  nine  heterogeneous  assets  with  mostly  sensors  and  decoys.  Wave  two, 
tasked  with  clearing  a  path  to  and  disabling  command  centers,  included  12  heterogeneous 
assets  with  sensors,  jammers,  and  shooters.  Interdiction  against  Time  Sensitive  Targets 
tasking  was  given  to  11  heterogeneous  assets  (sensors  and  shooters)  that  were  split 
evenly  by  capability  into  two  teams  searching  for  TELs  loosely  located  by  IPB 
information.  Blue  base  protection  tasking  was  given  to  the  six  remaining  assets  (  2  large 
sensors,  4  small  weapons  ).  These  were  allocated  to  CAP  missions  located  between  IPB 
indicated  "red  ground  forces  locations"  and  the  Blue  Base.  The  high  endurance  sensor 
platforms  were  tasked  to  stay  on  station  while  the  low  endurance  small  weapon  platform 
divided  with  two  on  station  and  the  remaining  two  as  replacements  remaining  at  the  base. 
Although  resource  constraints  prohibited  subsequent  execution  of  the  133rd  ANG 
strategy,  we  are  confident  that  their  approach  would  result  in  an  effective  strategy  for 
using  the  available  assets. 

A  significant  lesson  learned  is  the  importance  of  early  experimentation.  During  the  brief 
experimentation  conducted  we  were  able  to  work  with  the  researchers  to  achieve 
significant  improvement  in  their  capabilities.  We  are  confident  that,  if  more  time  had 
been  available,  they  would  have  achieved  considerable  success  in  executing  the  CP.  The 
original  MICA  program  plan  called  for  experimentation  in  the  4th  quarter  of  CY  02.  This 
experimentation  was  replaced  by  a  set  of  capabilities  demonstrations.  Although  these 
demonstrations  were  deemed  a  success  at  the  time,  they  eventually  proved  a  detriment  to 
the  program  when  viewed  in  the  context  of  premature  termination.  The  end  result  is  that 
because  experimentation  was  delayed,  researchers  failed  to  receive  valuable  feedback 
that  would  have  greatly  improved  their  end  products. 
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2  Introduction 

Teams  of  unmanned  platforms  operating  under  guidance  and  management  of  a  human 
operator  can  have  a  major  impact  on  achieving  battlefield  success.  The  MICA  program 
addressed  fundamental  technologies  for  control  of  teams  of  unmanned  platforms.  The 
MICA  vision  spans  the  waterfront  of  activities  required  for  development  and  transition  of 
these  technologies  to  military  forces.  It  includes:  1)  Composing  heterogeneous  teams  of 
heterogeneous  unmanned  platforms  with  mixed  weapon  and  sensor  packages  to  perform 
reconnaissance,  strike,  battle  damage  assessment  and  force  protection  activities;  2) 
Allocating  tasks  based  on  commanders  guidance  and  ROEs;  3)  Path  planning  activities 
for  teams  of  platforms  including  collision  avoidance  and  threat  avoidance/engagement;  3) 
Defining  effective  team  tactics  to  discover  and  attack  enemy  forces  using  real-time 
information;  and,  4)  Providing  information  to  human  operators  /  managers  sufficient  for 
effective  control  and  decision  making; 


MICA 

Technical  Decomposition 


Figure  2-1  -  MICA  Functional  Decomposition 


To  this  end,  the  MICA  Program  focused  effort  in  the  following  research  areas,  Figure  2- 
1: 

a.  Team  Composition  and  Tasking  (TCT) 

b.  Team  Dynamics  and  Tactics  (TDT) 
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c.  Cooperative  Path  Planning  (CPP) 

d.  Uncertainty  Management  (UM) 

e.  Variable  Initiative  Interaction  (VII) 


Specific  Goals  of  MICA  research  included:  1)  Achieve  M  operators  «  N  vehicles  (1:5 
by  2003  and  1:30  by  2005),  2)  Speed  the  sensor-to-shooter  cycle,  3)  Cooperatively 
couple  sensing  and  strike,  4)  Enable  flexible  self-reorganizing  teams,  and  5)  Allow  event- 
driven  dynamic  re-planning. 
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Figure  2-2  -  OEP  Overview 


The  MICA  Open  Experimentation  Platform  (OEP)  was  designed  to  provide  an 
experimentation  platform  for  simulation,  demonstration,  assessment  and  transition  of 
MICA  research  to  operational  systems.  The  OEP  also  responded  to  the  long-term  need 
by  incorporating  capabilities  to  support  the  broader  IXO  vision.  OEP  requirements 
therefore  cover  a  broad  spectrum  of  modeling  and  simulation  activities  required  for 
Finding,  Fixing,  Tracking,  Targeting,  Engaging  and  Assessing  (F2T2EA)  tactically 
significant  enemy  assets.  The  OEP  is  designed  to  provide  modeling  fidelity  appropriate 
to  MICA  requirements.  Because  the  OEP  is  not  designed  to  evaluate  lower  level  control 
schemes,  it  utilizes  point  mass  rather  than  six-degree-of-freedom  models.  OEP  models 
are  effects  based  when  feasible  and  provide  higher  fidelity  in  MICA  critical  areas  of 
concern.  Moreover,  the  OEP  deliberately  does  not  inherently  model  any  existing  or 
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planned  weapon  system  or  platform.  It  is  parameter  driven  and  for  MICA  parameters 
were  selected  to  stimulate  controller  development  and  ensure  robustness  in  response  to  an 
intelligent  adversary  in  a  balanced  scenario  (one  that  is  challenging  but  defeatable). 
Figure  2-2  shows  an  overview  of  the  OEP  application  relationship  to  MICA  research 
objectives. 

3  Accomplishments  and  Achievements 

The  guiding  principals  behind  our  OEP  Core  development  was  to  get  a  working  product 
into  the  researcher’s  hands  and  to  immediately  and  continuously  improve  this  product  in 
response  to  researcher  feedback.  In  response  to  these  needs  we  adopted  a  Spiral 
Development  approach  which  yielded  two  major  releases,  Versions  0.0  and  1.0,  and 
numerous  incremental  releases  in  the  period  from  October  2001  through  September  2003 
when  funding  for  the  program  was  curtailed. 

OEP  Version  0.0  was  released  in  December  2001,  approximately  60  days  after  contract 
award.  Version  0.0  provided  early  access  to  OEP  capabilities  in  order  to  enable 
researchers  to  familiarize  themselves  with  OEP  operations  and  to  initiate  early  feedback 
from  researchers  regarding  desired  changes  and  additional  capabilities.  Because  the  OEP 
is  built  upon  the  Boeing  C4I  Simulation,  which  has  been  under  development  and  in  use 
since  1994,  Version  0.0  provided  significant  capability  in  all  MICA  requisite  areas  of 
modeling  and  simulation.  An  extensive  presentation  was  presented  at  the  MICA  Kickoff 
Meeting  to  describe  OEP  Version  0.0  capabilities. 

Subsequent  to  release  of  Version  0.0,  Boeing  actively  sought  feedback  from  researchers 
regarding  their  requirements.  A  TIM  was  held  during  the  first  quarter  of  2002  to  review 
and  prioritize  researcher  requests.  Highest  priority  researcher  requests  included: 

•  Terrain 

•  Separate  locations  for  SAM  site  components  Sensor  misidentification 

•  Additional  API  operations 

•  Command  sensor  spotlight 

•  Get  target  identification  and  signal  properties 

•  Alter  vehicle  signature 

•  Max  range  for  weapon 

•  Jamming 

•  Update  OEP  scripts  dynamically 

•  Multiple  engagements  per  SAM  site 

•  Tracking  and  threat  warning  modes  in  sensor 

•  Push  interface  for  sensor  reports,  events 

•  Generalize  time  step  controller  for  use  by  all  components  /  controllers 

•  For  debugging,  event  playback  from  file 

•  Support  embedding  OpenMap  window  into  researcher  GUI 

•  Model  SAM  fly  out  and  terminal  maneuver 

These  requests  were  treated  as  candidate  requirements  which  were  prioritized  by  the 
MICA  Government  team  and  integrated  into  the  OEP  build  plan. 
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OEP  0.0  was  incrementally  improved  (six  releases)  over  the  period  between  December 
2001  and  May  2002.  The  culmination  of  theses  incremental  releases,  OEP  Version  1.0, 
was  released  in  May  2002.  OEP  1.0  responded  to  all  but  the  last  four  of  the  researcher 
feedback  items  presented  in  the  previous  paragraph.  Major  OEP  1.0  capabilities 
included:  1)  capability  to  locate  objects  over  a  terrain  grid  and  execute  inter- visibility 
analyses  between  airborne,  space-based  and  ground  objects,  2)  capability  to  compose 
SAM  sites  from  multiple,  separated  elements  including  search/acquisition  radar,  tracking 
(fire  control)  radar,  controller  and  multiple  launchers,  3)  Sensor  spotlight  API  call  that 
causes  a  multi-mode  sensor  to  interrupt  search  scan  to  perform  a  spotlight  cycle  at  a 
specified  geographic  location.  4)  Small  arms  model  reflecting  effect  of  shoulder  fired 
SAMs,  5)  Dynamic  network  configuration,  6)  Sensor  identification  of  targets,  7)  position 
error  in  weapon  aim  point,  8)  option  to  set  and  vary  platform  signature,  and  9)  platforms 
may  be  equipped  with  jammers  that  work  against  radar  and  passive  RF  sensors. 

Subsequent  to  delivery  of  OEP  1.0,  Boeing  OEP  core  activities  concentrated  on 
additional  capabilities  desired  by  researchers  to  support  the  October  2002 
Demonstrations.  Seven  incremental  releases  (OEP  1.0.1  through  1.0.7)  added  these 
capabilities  including  software  hardening  in  Build  1.0.1.  OEP  capabilities  incorporated 
into  Build  1.0.1  included:  (1)  Event  notification  with  buffered  events;  (2)  GMTI  sensor 
mode;  (3)  Imaging  sensor  mode;  (4)  Target  classification;  (5)  Limitations  on  platform 
motion;  (6)  Scenario  inputs;  (7)  Additional  information  query;  and  (8)  Multiple  buffered 
event  subscription. 

In  the  period  between  October  2002  and  September  2003,  Boeing  produced  three 
additional  major  OEP  increments.  The  focus  of  build  1.1  was  to  incorporate  additional 
modeling  capabilities  to  support  more  realistic  challenge  problems  and  scenarios.  Of 
primary  interest  was  OEP  support  for  cooperative  path  planning.  OEP  1.1,  released  in 
Feb  2003,  provided  several  new  capabilities:  (1)  3D  platform  signatures  in  several 
spectral  bands;  (2)  An  image  interpreter  can  process  multiple  images  concurrently;  (3) 
Weapon  guidance  modes  for  seeker  and  RF  seeker  weapons  were  added;  (4)  Platforms 
may  employ  active  signature  enhancement;  (5)  Side  lobe  jamming  may  be  employed;  (6) 
Synthetic  aperture  radar  imaging  sensors  are  susceptible  to  jamming;  (7)  Air  search 
radars  and  passive  RF  sensors  now  report  jamming  strobes;  (8)  Damage  and  destroy 
probabilities  vary  by  distance  and  the  cumulative  number  of  damaging  hits;  (9)  Terrain 
line  of  sight  test  uses  nearest  post  rather  than  interpolation,  reducing  run  time;  (10) 
Terrain  line  of  sight  test  was  optimized  to  start  at  the  lower  point  and  stop  when  highest 
scenario  altitude  reached;  (11)  Platforms  without  emitters  exit  earlier  from  the  passive  RF 
sensor  detection  function,  reducing  run  time;  (12)  Event  subscription  by  CORBA  objects 
in  addition  to  XML;  (13)  Image  events  can  be  obtained  by  subscribing  either  to  the  image 
sensor  platform  or  to  the  image  interpreter  platform;  (14)  Small  arms  threats  may  be 
limited  to  specific  geographic  areas  ;  (15)  Additional  buffered  events  were  added;  and 
(16)  Additional  OEP  server  calls  were  added.  In  addition  a  preprocessor  was  added,  and 
the  challenge  problem  was  divided  into  modular  files.  The  red  controller  will  try  to  avoid 
engaging  targets  that  appear  to  be  decoys  (the  use  of  active  signature). 
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The  focus  of  OEP  1.2  was  to  support  MICA  controller  development  of  team  tactics.  OEP 
1.2,  released  April  2003,  included:  (1)  The  jammer  model  was  revised  to  support  team 
jamming  by  adding  jammer  power  from  all  sources,  (2)  A  multilateration  model  was 
added  to  support  team  position  location  of  emitters,  (3)  Passive  RF  sensors  can  be 
connected  to  a  multilateration  component  in  a  platform  rather  than  to  a  track  list,  (4) 
Track  list  correlation  permits  miscorrelation,  (5)  A  tracker  was  added  to  the  location 
update  function  in  track  lists,  (6)  Weapon  carrying  load  limits  were  implemented,  and  (7) 
the  defense  (SAM)  model  was  changed  to  add  more  states  and  events  for  the  red 
controller. 

The  focus  of  OEP  1.3  was  to  provide  support  of  mid-term  software  and  hardware 
experimentation.  OEP  1.3,  released  in  mid- August,  provided  several  new  capabilities:  1) 
The  initial  scoring  and  metrics  interface,  that  enables  clients  to  call  the  OEP  server  and 
obtain  the  current  metrics  that  will  be  used  in  the  experimentation  phase  to  evaluate 
controllers,  was  added;  2)  The  TasklmagingSensor  operation  was  changed  to  add  priority 
and  request  id  parameters;  3)  The  TaskGMTISensorList  server  operation,  which  enables 
tasking  a  GMTI  sensor  with  a  list  of  cells  rather  than  with  a  rectangular  region  of 
contiguous  cell,  was  added;  4)  The  GetGMTIProperties  and  GetlmagingProperties  server 
operations  were  added  to  retrieve  sensor  parameters  that  are  useful  for  planning  sensor 
tasks;  5)  In  platform  types,  the  maxverticalrate  parameter  was  replaced  by 
max_climb_rate  and  maxdescentrate,  and  a  max_alt  parameter  was  added  for  future 
use;  6)  In  the  scenario/outputs/scenario/print_format  section  of  the  scenario  input  the 
show_zero  parameter  that  was  in  the  classification  and  damage  parameter  sections  was 
changed  to  show_small_prob;  7)  The  OEP  now  throws  an  exception  if  there  is  a  platform 
source  conflict  in  a  subscription;  8)  The  IADS  model  can  now  randomly  choose  to 
engage  the  target  with  the  best  side  view  (for  higher  signature)  in  addition  to  the  previous 
tactic  of  selecting  the  closest  target;  9)  There  is  now  a  limit  on  the  number  of  SAM  sites 
that  be  moving  at  any  one  time;  10)  Ground  coverage  sizes  for  imaging  sensors  may  be 
set  by  inputs;  11)  Support  for  time-varying  commander’s  guidance  and  rules  of 
engagement  was  added;  12)  A  scripted  platform,  such  as  the  blue  base,  is  allowed  to 
broadcast  messages  to  controller  Proxies;  13)  The  obsolete  sensor  spotlight  mode  and  the 
server  operation  AddSpotLightRequest  were  removed;  and  14)  The  drop  age  parameter 
was  moved  from  platform  type  to  track  list,  where  values  may  be  specified  by  motion 
hint. 

In  order  to  stimulate  operationally  relevance,  a  rolling  series  of  challenge  problems, 
formulated  to  stress  research  objectives,  were  developed  in  conjunction  with  the  OEP 
core  capabilities.  The  challenge  problems  posed  formidable  sensor  management  issues 
for  finding  and  attacking  targets  and  stressed  the  need  for  resource  planning  and  team 
operations.  The  Challenge  Problem  scenario  encompassed  a  representative  Red  Order  of 
Battle  including  an  Integrated  Air  Defense  System  and  Time  Critical  Targets  including 
both  Surface-to-Surface  Missiles  and  Armored  Columns,  which  threatened  Blue  Assets. 
It  also  included  both  fixed  and  moving  white  objects  that  presented  potential  for 
collateral  damage.  Figure  3-1  shows  an  overview  of  the  MICA  Challenge  Problem.  The 
challenge  problem  is  scalable  to  larger  number  of  UAVs,  dynamic  in  nature  with 
changing  objectives  and  incorporates  random  disturbances  provide  system  shocks. 
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Challenge  Problem  Overview 

•  Red  Order  of  Battle  Elements 

♦  IADS  with  C2,  ESM,  EW,  Long  and 
Medium  SAMS 

♦  Roads  with  white  and  red  traffic 

+  Polygonal  boundaries  for  red 

concentrations,  kill,  and  restricted 
zones 

♦  Mobile  armor  and  TBM  launchers 

♦  Red  small  arms  provides  attrition  at 
low  level 

♦  Cultural  features  for  collateral 
damage 

•  Blue  Unmanned  Platforms 

♦  Heterogeneous  mix  including  sensor, 
weapon,  and  combo  platforms 

♦  Multiple  weapon  types 

+  GMTI,  S  AR,  ESM  sensors 

♦  Jammers  &  Decoys 

♦  Launch  detection  sensor 


Figure  3-1  -  Challenge  Problem 


4  Methods,  Assumptions  and  Procedures 

The  MICA  OEP  is  an  effects-based,  campaign-level  simulation  that  produces  two  views 
of  the  battlespace:  truth  and  perception  (from  multiple  viewpoints).  In  order  to  model 
campaign-level  effects,  the  OEP  is  designed  to  efficiently  simulate  large  numbers  of 
platforms  and  targets.  By  Effects  Based,  we  mean  that  the  effects  of  phenomena  and 
actions  are  modeled  rather  than  the  detailed  physics.  The  simulation  normally  utilizes  a 
simple  vehicle  motion  model  but  can  also  interface  to  higher  fidelity  third  party  motion 
models.  It  emphasizes  fidelity  in  areas  that  are  important  to  the  MICA  problem  space.  In 
particular,  sensor  errors  and  the  propagation  of  errors  from  producer  to  consumer  are 
treated  in  substantial  detail,  as  are  communication  network  delays  and  throughput. 
Sensor  model  fidelity  is  enhanced  by  a  three-dimensional  vehicular  signature  capability 
that  can  be  tailored  to  each  sensors  spectral  domain.  The  OEP  is  normally  employed  in  a 
Monte  Carlo  fashion  in  order  to  achieve  reasonable  measures  of  performance.  Users  may 
specify  random  distributions  or  constant  values  for  the  random  variables.  Simulation 
entities  are  customizable  and  may  represent  a  variety  of  air  and  ground  vehicles,  targets, 
and  facilities. 
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Architecture 


Scenario  object 


Platform  object 

connectable  components 

comm  ports 

emitters 

sensors 

track  lists 

proxies 

defensive 

image 

|  interpreters 

weapon 

multi- 

lateration 

non-connectable 

offensive 

fuel 

weapons 

script 

Network  object 


channels 


•  A  scenario  contains  properties  and 
collections  of  platforms  and  networks 

•  Platforms  and  networks  each  contain 
properties  and  components 

•  User  input 

♦  Selects  and  connects  platform 
components 

♦  Connects  platform  comm  ports  to 
networks 


0LJ 


Figure  4.1-1  -  OEP  Architecture 


Component  connections 


Figure  4.1-2  -  OEP  Component  Connections 
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4.1  Architecture 

The  OEP  Architecture,  Figure  4.1-1,  relies  on  two  major  simulation  entities  -  platforms 
and  networks.  Platforms  are  constructed  from  parameterized  building  blocks,  which  are 
connected  to  each  other  by  user  input  parameters.  Connectible  platform  components 
include:  1)  Communication  Ports,  2)  Emitters,  3)  Sensors,  4)  Track  Lists,  5)  Proxies,  6) 
Defensive  Weapons,  7)  Image  Interpreters  and  8)  Multilateration  Cells.  Non-connectible 
components  include:  1)  Offensive  Weapons,  2)  Fuel  and  3)  Scripts.  Networks  may  be 
customized  to  represent  point-to-point,  multi-point,  half  duplex,  full  duplex,  broadcast, 
and  time  division  multiple  access  networks.  Network  connections  among  platforms  are 
specified  by  user  input  parameters.  Figure  4.1-2  shows  typical  connectivity  between 
platform  elements  using  OEP  Network  Models.  The  OEP  User’s  Manual  provides  details 
of  all  Platform  and  Network  components. 


Configurable  platforms 

•  To  the  software,  all  entities  are 
platforms 

•  User  input  defines  a  platform's  role 
and  behavior 

•  Motion 

♦  Straight  line  segments 

•  Signature 

•  Reliability 

•  Component  functions,  connections 


Aircraft 

Helicopters 


Ground  and  air  targets 
Tanker  aircraft 
Command  centers 
Radar  sites 
Tanks  and  trucks 
SAM  sites 
Ships 
Satellites 
Other  things 


Figure  4.2-1  -  Configurable  Platforms 


4.2  Platforms 

Platform  objects,  Figure  4.2-1,  can  be  used  to  model  numerous  battlefield  entities 
including:  Aircraft,  Rotorcraft,  Ground,  naval  and  air  targets,  Tankers,  Command  and 
Control  elements,  Radar  sites,  Armored  vehicles  including  tanks  and  Self  Propelled 
Artillery  (SPARTY),  SAM  sites,  Satellites,  etc. 
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4.2.1  Platform  Motion 

The  OEP  includes  three  basic  models  for  platform  motion:  1)  Acceleration  motion 
model;  2)  Rotorcraft  motion  model;  and  3)  External  6-DOF.  In  the  acceleration  model 
we  use  a  flight  dynamics  model  with  3  degrees  of  freedom  and  a  simple  3-channel 
autopilot  that  accepts  commanded  speed,  acceleration  limits,  and  either  a  destination 
location  or  commanded  altitude  and  heading.  Gains  and  limits  are  specified  in  the  input. 
The  autopilot  feedback  control  loops  calculate  acceleration  commands  that  are 
numerically  integrated  to  get  velocity  and  position.  Body  yaw  angle  and  heading  are  set 
to  the  flight  path  azimuth  angle.  Bank  angle  is  synthesized  from  lateral  and  vertical 
acceleration.  A  hold  position  command  causes  the  vehicle  to  fly  in  a  circle. 

In  the  rotorcraft  motion  model  we  use  a  flight  dynamics  model  with  3  degrees  of  freedom 
and  a  simple  3 -channel  autopilot  that  accepts  commanded  speed,  acceleration  limits,  and 
either  a  destination  location  or  commanded  altitude  and  heading.  Gains  and  limits  are 
specified  in  the  input.  The  autopilot  feedback  control  loops  calculate  acceleration 
commands  that  are  numerically  integrated  to  get  velocity  and  position.  Bank  angle  is 
synthesized  from  lateral  and  vertical  acceleration  components.  Pitch  angle  is  synthesized 
to  simulate  the  effect  of  the  cyclic  control.  Body  yaw  angle  can  be  commanded  separately 
from  heading.  A  hold  position  command  causes  the  vehicle  to  hover. 

The  OEP  also  includes  an  interface  to  external  six  degree  of  freedom  model.  This 
enables  OEP  to  utilize  an  external  simulation  to  provide  high  fidelity  vehicle  dynamics. 
One  instantiation  of  this  is  in  the  use  of  the  Georgia  Institute  of  Technology  model  for  the 
Yamaha  R-MAX.  A  software  interface  adapts  the  Georgia  Tech  code  to  the  simulation 
pluggable  motion  interface. 

4.2.2  Signature 

For  the  3D  Signature  Capability,  platform  signature  can  vary  with  azimuth  and  elevation 
angles.  Signature  determination  can  be  performed  using  the  nearest  point  in  the 
Azimuth/Elevation  Table  or  through  linear  interpolation.  The  signature  is  an  arbitrary 
numeric  value,  which  is  associated  with  sensor  performance  through  table  lookups. 
Unique  signatures  can  be  associated  with  each  spectral  band  (RF,  visible,  etc.).  Separate 
signature  tables  can  be  provided  for  different  platform  states  (damaged,  bay  doors  open, 
etc).  Azimuth  values  can  be  symmetric  or  asymmetric  about  body  axis.  For  the  purpose 
of  determining  signature,  Azimuth  =  heading,  Elevation  =  flight  path  angle  and  Bank  = 
+45  deg  if  turning,  0  otherwise. 

4.3  Sensors 

The  sensor  model  provides  general  functions  to  simulate  these  kinds  of  sensors:  radar, 
radar  with  ground  moving  target  detection  (GMTI),  imaging  (visible,  infrared,  synthetic 
aperture  radar  (SAR),  and  radio  frequency  signal  measurement  (ELINT  or  ESM).  There 
are  no  specific  sensor  models  in  the  software.  Models  of  specific  sensors  are  formed  with 
input  parameters.  Figure  4.3-1  shows  the  general  processing  steps  in  sensors.  The  labels 
indicate  the  steps  that  apply  only  to  certain  types  of  sensors.  Although  the  image 
interpreter  is  technically  not  a  sensor  or  sensor  mode  in  the  software,  it  is  shown  in  the 
diagram  because  it  is  closely  related  to  the  imaging  sensor. 
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Interfaces 


Figure  4.4-1  -  OEP  Interfaces 
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4.4  Interfaces 

Figure  4.4-1  shows  the  MICA  OEP  Interfaces.  The  OEP  Core  Server,  Simulation 
Models,  provide  a  CORBA  Application  Program  Interface  (API)  to  multiple  clients.  The 
Challenge  Problem  models  and  scenario  are  executed  using  the  Script  Client.  The  Map 
Client  provides  a  graphic  interface  (currently  Open  Map)  to  help  visualize  scenario 
progress.  Researcher  Blue  Controllers,  the  Red  Integrated  Air  Defense  System  (IADS) 
controller  and  the  planned  Red  Reactive  Ground  controller  use  this  interface  along  with 
other  clients  including  Data  Collection  and  Run  Control.  The  Console  Application 
allows  the  OEP  to  be  run  in  a  batch  mode  to  assist  in  scenario  development,  debugging 
and  test. 


4.5  Red  Controller  Clients 


Integrated  air  defense _ 

•  Command  center  track  file  merges  reports 
from  networked  search  radars 

•  lADS-controlled  SAMs  do  not  radiate  until 
target  assigned 

•  Command  center  tasks  SAM  sites,  based 
on: 

♦  Not  at  max  engagements 

♦  Has  missiles 

♦  Closest 

•  Redundant  command  centers 

•  Command  center  destroyed 

♦  Automatically  switch  to  backup  command 
center 


♦  If  all  command  centers  destroyed,  SAMs 
switch  to  autonomous  operation 
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Figure  4.5-1  -  Integrated  Air  Defense  Capabilities 


The  MICA  OEP  also  includes  a  Red  Controller  Client,  which  converts  selected  Red  air 
defense  elements  into  an  Integrated  Air  Defense  System  (IADS).  Within  the  IADS, 
designated  EW  Radars,  SAM  Search  Radars  and  ESM  sensors  contribute  to  an  integrated 
red  intelligence  database.  The  IADS  controllers,  Located  at  Red  Command  and  Control 
(C2)  facilities  control  actions  executed  by  the  integrated  red  SAMs.  If  all  red  (C2) 
facilities  are  destroyed  the  Red  SAMs  revert  to  independent  operation.  The  IADS 
provides  red  defenses  with  the  ability  to  radiate  only  after  they  are  commanded  to  engage 
targets  that  are  well  within  their  lethal  envelope.  The  engaging  SAM  is  selected  based  on 
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criteria  including:  Number  of  missiles  remaining,  Proximity  to  approaching  blue  asset, 
and  blue  geometry  (signature)  relative  to  SAM.  The  IADS  also  includes  a  “Shoot  and 
Scoot”  capability  that  relocates  SAMs  to  alternative  pre-planned  locations  after  they  have 
disclosed  their  location  through  emission.  The  “Shoot  and  Scoot”  logic  moves  SAMs 
only  when  engagements  are  not  imminent.  The  logic  also  controls  the  maximum  number 
of  SAMs  in  transit  at  any  time.  Figure  4.5-1  highlights  OEP  IADS  capabilities. 

5  Results  and  Discussion 

MICA  research  was  pre-maturely  terminated  due  to  evolving  priorities  within  DARPA. 
Only  a  subset  of  the  available  controllers  was  submitted  to  Boeing  for  evaluation  and 
those  controllers  submitted  were  less  capable  than  would  have  been  the  case  if  planned 
research  had  been  completed.  Controllers  submitted  to  Boeing  were  produced  by  Draper 
Laboratories  and  by  the  collaborative  efforts  of  Ohio  State  University  and  Iterativity  Inc. 
Two  variants  of  the  Iterativity-OSU  controller  were  provided.  One  uses  the  Ultra  planner 
and  the  other  uses  the  Hierarchical  planner.  In  general  our  discussion  will  treat  the 
Iterativity-OSU  controller  as  a  single  product.  Where  differences  between  the  Ultra  and 
Hierarchical  planners  are  of  note,  they  will  be  explicitly  discussed.  Evaluations  of  both 
controllers  were  performed  at  the  Boeing  St  Louis  facility  during  the  fourth  quarter  of 
CY03  and  the  first  quarter  of  CY04.  Both  research  teams  were  provided  feedback 
regarding  observed  behavior  and  were  afforded  multiple  opportunities  to  explain  and  or 
correct  deficiencies.  Considerable  progress  and  improvement  occurred  during  the 
evaluation  period.  Unfortunately,  neither  controller  was  ultimately  capable  of 
demonstrating  acceptable  tactical  utility  when  evaluated  against  the  MICA  challenge 
problem.  This  lack  of  comprehensive  progress  may  well  be  attributed  to  the  abrupt 
termination  of  the  development  cycle  that  left  many  algorithmic  loose  ends.  It  is  also 
possible  that  a  fundamental  lack  of  maturity  exists  in  one  or  more  of  the  constituent 
technologies. 

5.1  Summary  of  Results 

Despite  the  early  termination,  much  was  accomplished.  Both  controllers  exhibited  a 
hierarchical  decomposition  of  the  challenge  problem  and  allocated  teams  of  assets  to 
subsets  of  known  targets.  Both  controllers  evidenced  coordinated  team  tactics  and 
planned  asset  flight  paths  in  response  to  either  team  assignment  or  task  allocation. 
Provisions  were  put  in  place  to  address  the  inherent  uncertainty  of  knowledge  of  the 
battlespace  such  as  Draper's  Information  Model  and  Iterativity's  VII IPB  interface.  Both 
controllers  have  a  broad  Variable  Initiative  Interface  (VII),  albeit  of  varying  levels  of 
maturity  and  utility.  Unfortunately  there  was  also  much  left  undone.  The  following 
summarizes  observed  behavior  in  each  of  the  research  areas  as  well  as  certain  functional 
categories.  It  is  followed  by  a  more  detailed  discussion  organized  by  MICA  research 
areas. 

Team  Composition  and  Tasking  -  The  construct  of  team  in  the  context  of  the  MICA 
controllers  appears  more  to  be  simply  a  product  of  the  logical  partitioning  the  challenge 
problem  for  planning  simplification,  rather  than  the  grouping  of  assets  that  dynamically 
and  purposefully  cooperate.  All  intra-team  coordination  is  the  consequence  of 
centralized  a  priori  action  scheduling  or  sequencing. 
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Team  Dynamics  and  Tactics  -  All  controllers  have  displayed  some  level  of  team  tactics: 
cooperative  jamming,  bomb  damage  assessment,  and  the  explicit  coordination  of  actions 
performed  by  individual  team  assets.  Unfortunately,  the  lack  of  an  explicit  logical 
communication  activity  functionally  prevents  any  dynamic  cooperation  among  teamed 
assets.  There  is  no  "active"  cooperation,  all  coordinated  activities  are  simply  the  product 
of  a  priori  plan  sequencing,  and  unless  an  existing  plan  is  centrally  altered,  the  planned 
tactical  activities  will  be  executed  regardless  of  whether  or  not  all  constituent  team 
members  are  present. 

Cooperative  Path  Planning  -  Both  global  and  local  path  planners  were  implemented, 
and  all  of  the  controllers  did  plan  flight  paths  according  to  either  team  assignment  or  task 
allocation.  Unfortunately,  all  of  the  path  planners  apparently  reasoned  primarily  within  a 
horizontal  plane.  That  is,  none  of  the  controllers  rigorously  planned  using  the  three 
dimensional  platform  signatures,  which  if  exploited,  would  have  significantly  increased 
asset  survivability  while  navigating  a  SAM  engagement  zone. 

Uncertainty  Management  -  The  Draper  Information  Model  provides  a  natural 
representation  for  both  the  'certain'  knowledge  of  the  world,  such  as  active  target  tracks, 
and  the  corresponding  'uncertain'  awareness  of  expected  but  as  of  yet  un-discovered 
targets.  And  in  fact,  both  of  these  types  of  information  are  provided  by  the  MICA  IPB. 
Draper  currently  populates  the  Information  Model  with  “truth"  IPB  data,  extracted  from 
the  scenario  XML  fde,  rather  than  using  the  IPB  provided. 

Variable  Initiative  Interaction  -  The  purpose  of  the  VII  is  to  provide  the  MICA 
Operator  with  sufficient  information  to  develop  the  situational  awareness  necessary  to 
interact  with  and  control  heterogeneous  teams  of  UAVs.  Even  though  the  VIIs  allowed 
the  operator  to  see  a  god's  eye  view  of  the  situation,  input  various  planning  parameters, 
and  review  the  generated  plans,  there  was  no  direct  means  to  communicate  new  Strategic 
Objectives,  Commander's  Guidance,  or  Rules  of  Engagement.  Moreover,  none  of  the 
VIIs  actually  allowed  the  MICA  Operator  to  directly  interact  with  the  planning  process. 
No  VII  allowed  the  operator  to  edit  a  plan,  either  at  the  team  composition  level,  or  at  the 
task  allocation  level.  The  operator  could  only  initiate,  accept,  or  reject  a  plan.  Providing 
this  ability,  as  well  as  a  mean  to  visualize  and  implement  higher-level  strategies,  would 
greatly  enhance  the  utility  of  the  VII. 

Action  Scheduling  vs.  Action  Sequencing-  There  are  two  fundamental  approaches  to 
structuring  the  "actionable"  plans,  action  scheduling  or  action  sequencing.  The  Draper 
controller  planned  for  asset-level  cooperation  by  deterministically  scheduling  all 
activities,  whether  coordinated  or  not,  in  a  global  timetable.  Alternatively,  the  Iterativity- 
OSU  controllers  planned  for  cooperation  by  establishing  precedence  and  concurrence 
relationships  among  only  those  activities  that  are  to  be  coordinated,  thus  placing  fewer 
constraints  upon  the  final  plan  that  can  be  broken  by  an  un-anticipated  turn  of  events. 

Thus  the  Draper  controller  is  schedule  driven  and  not  event  driven.  An  asset  will  simply 
perform  an  action  for  what  has  been  determined  is  the  appropriate  amount  of  time,  and 
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then  wait  until  it  is  time  to  begin  the  next  scheduled  action.  A  benign  example  of  this  is 
planned  path  execution.  Rather  than  an  asset  simply  flying  until  receipt  of  a  waypoint 
arrival  event,  the  controller  will  "fly"  for  the  pre-determined  time  interval,  and  then  query 
the  OEP  for  platform  location  to  determine  arrival  status.  A  less  benign  example  is  when 
the  controller  performs  a  BDA  imaging  action.  After  the  execution  of  a  sensor  imaging 
action,  rather  than  wait  for  receipt  of  an  "image  interpreter  report  event",  the  controller 
waits  for  a  predetermined  amount  of  time  under  the  assumption  that  the  new  image  data 
will  have  been  "analyzed  by  the  image  interpreters"  and  incorporated  into  the  central 
Information  Model.  While  this  approach  will  work  much  of  the  time,  it  cannot  account 
for  indeterminate  image  interpreter  delays  that  may  result  from  network  congestion  or 
from  processing  an  unknown  number  of  images  generated  by  other  assets. 

Alternatively,  the  Iterativity-OSU  controllers  simply  sequence  the  planned  activities,  and 
then  repeatedly  test  for  activity  completion.  This  allows  for  'less-brittle'  plans  since 
fewer  constraints  need  to  be  established  which  might  later  be  violated.  For  example, 
perturbing  a  "FlyTo"  activity  to  avoid  a  discovered  threat  does  not  necessarily  invalidate 
the  larger  plan.  Since  the  next  action  is  minimally  constrained  to  begin  upon  completion 
of  the  prior  action,  it  will  simply  begin  a  bit  later  than  it  otherwise  would. 

In  general,  it  appears  that  action  scheduling  tends  toward  more  brittle  plans  that  can  more 
easily  be  broken  when  events  do  not  occur  as  anticipated.  Alternatively,  action 
sequencing  appears  to  be  more  flexible  by  logically  allowing  for  unanticipated  delays 
from  an  uncertain  adversarial  environment. 

Event  Handling-  There  is  15  event  types  available  to  the  controllers  from  the  MICA 
OEP.  Thirteen  of  these  event  types  directly  represent  either  state  information  of  the 
platform  or  state  of  the  world,  or  are  the  results  of  platform  actions.  OEP  event  types 
available  for  registration  are 


received  communication 

sensor  collection  start 

sensor  collection  done 

platform  condition 

sensor  collection  reported 

sensor  report 

waypoint  arrival 

image  interpreter  report 

multilateration  report 

weapon  release 

track  report 

weapon  arrival 

threat  warning 

Of  the  15  available  event  types,  the  MICA  controllers  have  registered  for  the  following 
subset  of  events,  and  appear  to  actively  "consume"  even  less  . . . 

SEN  S  ORCOLLECTIONST  ART  OSU 

SENSORCOLLECTIONDONE  OSU  Draper 

THREATWARNING  Draper 
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TRACKREPORT  OSU  Draper 

The  only  event  type  determined  with  certainty  to  be  consumed  by  the  controllers  and 
thereafter  used  for  planning  is  the  track  report.  This  artificial  limit  on  the  type  of  events 
consumed  will  arguably  limit  how  the  controllers  represent  this  "knowable"  yet  ignored 
state  of  the  word.  And  moreover,  this  will  limit  with  what  and  how  the  controllers  react 
in  the  world.  For  example,  it  is  the  threat  waning  event  that  conveys  perhaps  the  most 
urgent  information  about  the  state  of  the  world  and  immediate  intent  of  the  enemy,  that  a 
SAM  is  actively  engaging  a  blue  asset  with  the  intent  of  destroying  it.  Unfortunately,  the 
Iterativity-OSU  controllers  did  not  register  for  threat  warning  events,  and  even  though  the 
Draper  controller  did,  there  was  no  evidence  that  they  made  use  of  the  threat  warning 
reports  in  their  planning  or  world  model. 

Events  are  retrievable  from  the  MICA  OEP  in  both  a  "batch"  mode  or  upon  specific  event 
occurrence,  but  in  both  cases,  event  retrieval  must  be  initiated  by  the  OEP  client.  In  the 
beginning  of  experimentation,  the  Draper  controller  only  retrieved  events  at  the  end  of  a 
'planning  increment',  which  could  be  on  the  order  of  several  hundred  seconds  simulation 
time.  In  order  for  a  controller  to  react  to  the  dynamic  adversarial  environment,  the  event 
query  period  should  be  less  than  the  expect  threat  episode,  which  for  a  medium  SAM  is 
about  25  seconds.  Additionally,  these  extended  retrieval  periods  would  often  lead  to 
event  queues  containing  several  thousand  events  to  be  processed.  By  the  end  of  the 
experimentation  phase,  all  controllers  were  querying  for  events  on  the  order  of  every  10 
seconds,  but  no  controller  was  yet  making  use  of  the  threat  warning  events. 

In  general,  controller  performance  could  be  fundamentally  improved  by  more  frequent 
consumption  of  a  larger  set  of  event  types  that  would  provide  a  richer  representation  of 
the  dynamic  and  adversarial  state  of  the  world. 

5.2  Detailed  Discussion 

MICA  experimentation  was  designed  to  assess  research  artifacts  (controllers)  to  quantify 
their  capability  in  each  of  the  five  research  areas: 

1)  Team  Composition  and  Tasking, 

2)  Team  Dynamics  and  Tactics, 

3)  Cooperative  Path  Planning, 

4)  Uncertainty  Management, 

5)  Variable  Initiative  Interaction, 

and  to  measure  their  robustness  in  adapting  to  a  variety  of  tactical  situations: 

a.  SEAD  against  individual  SAM  sites, 

b.  SEAD  against  an  Integrated  Air  Defense  System, 

c.  Interdiction  against  fixed  and  moving  targets, 

d.  Intelligence,  Surveillance,  and  Reconnaissance  (ISR), 

e.  Employment  of  a  variety  of  weapons,  sensors,  and  countermeasures, 
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f.  Adherence/adaptability  to  the  Commander's  Guidance  and  Rules  of 
Engagement. 

The  remainder  of  this  section  further  discusses  experimentation  results  categorized  in 
each  of  the  five  research  areas. 

5.2.1  Team  Composition  and  Tasking 

The  MICA  operational  vision  entails  "teams  of  heterogeneous  UAVs  with  mixed  weapon 
and  sensor  packages  performing  reconnaissance,  strike,  BDA  and  force  protection 
activities"  under  the  supervision  of  a  few  (ultimately  1  operator  per  40  vehicles)  human 
operators.  All  of  the  controllers  evaluated  exhibited  a  hierarchical  decomposition  of  the 
challenge  problem  and  allocated  teams  of  assets  to  subsets  of  known  targets.  The 
optimality,  responsiveness  and  sophistication  of  this  allocation  are  discussed  below. 

5.2.1. 1  Team  Composition 

The  construct  of  teams  by  the  evaluated  controllers  appears  to  be  a  partitioning  of  assets 
for  planning  simplification  rather  than  a  grouping  of  assets  that  purposefully  cooperate  to 
achieve  tactical  objectives  IAW  Commander’s  Guidance.  The  coordination  of  assets  is 
achieved  by  a  central  planner  rather  than  through  dynamic  distributed  or  centralized 
logic.  Specifically  assigned  tasks  are  pre-arranged  in  time  either  by  sequence  or  by 
schedule.  The  planners  either  establish  an  ordering  or  precedence  constraints  for  all 
tasks,  or  assign  a  start  time  and  duration  for  each  task. 


* 

Team  2-13  MINI  1  1 

T 

1  1  1  1  1  1  1  1  1  1 

]□□  rr 

mnn  rr 

“no  rr 

“nomoo 

■k 

Team  2-21  I  I  I  I  I  I  I 

1 

1  1  1  1  1  1  1  1  1  1 

]□□  LL 

i  i*ni  ii 

i  in  ii 

iiimin 

X 

Team  2-33  1  1  1  1  1  1 _ | _ |_ 

J 

i  i  i  i  i  i  i  mininrr 

i  i  mill 

J _ 1 1 1  ILL 

uomoo 

Figure  5.2.11-1  -  Sample  Draper  Team  Composition 


In  most  cases,  teams  were  constructed  as  a  mix  of  heterogeneous  platform  types  having 
complementary  sensing,  strike,  and  defensive  capabilities.  The  Draper  controller  would 
often  plan  around  small  teams  composed  of  different  platform  types,  such  as  the  team 
below,  Figure  5.2.1. 1-1, 
composed  of  one  sensor 
platform,  one  weapon 
platform,  and  one  combo 
platform. 


The  Iterativity-OSU 

controller  composed  fewer, 
but  larger  more  stable  teams. 

As  with  the  Draper 

controller,  these  teams  were  _.  _  ^  ^  „  ,  .  .  _ _  .  .... 

composed  of  a  mix  of  Figure  5.2.1.2-1  -  Iterativity-OSU  Planning  Activities 

weapon  and  sensor  platform 
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types,  as  is  illustrated  below  with  a  snippet  from  an  OSU  TCT  planner  log  file.  Neither 
controller  exhibited  a  strong  coupling  between  the  tactical  situation/objectives  and  the 
composition  of  assigned  teams. 


stages uaviNamesini  earns  laoieLujLujLuj  -  sman_weapon_i 
stagesUavNamesInTeamsTable[0][0][l]  =  small_weapon_2 
stagesUavNamesInTeamsTable[0][0][2]  =  small_weapon_3 
stagesUavNamesInTeamsTable[0][0][3]  =  small_weapon_4 
stagesUavNamesInTeamsTable[0][0][4]  =  small_sensor_2 
stagesUavNamesInTeamsTable[0][0][5]  =  small_sensor_l 
stagesUavNames!nTeamsTable[0][0][6]  =  small_combo_l 


5.2. 1.2  Activities  Planned 

Iterativity-OSU  defined  seven  distinct  activity  types  that  can  be  reasoned  over  and 
planned  with  as  illustrated  in  Figure  5. 2. 1.2-1.  The  seven  activity  types  represent  all 
fundamental  platform  capabilities  except  communication  and  controlled  network 
connection  required  for  multilateration  team  formation.  Figure  5. 2. 1.2-2  is  a  plan 
sequence  illustrating  five  of  the  logical  activities.  No  OSU-Iterativity  plans  have  been 
observed  which  contained  either  the  'refueling'  or  force  'protection'  activities. 
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Figure  5.21.2-2  -  Iterativity-OSU  Plan  Composition 

The  Draper  controller  plans  were  constructed  with  10  distinct  activity  types,  six  of  which 
are  illustrated  in  Figure  5. 2. 1.2-3.  Again  these  activity  types  represent  the  fundamental 
platform  capabilities  except  communication  and  controlled  network  connection. 


Figure  5.2.1.2-3  -  Draper  Plan  Composition 

This  lack  of  a  logical  communication  activity  type  to  reason  upon  prevents  a  plan  from 
containing  dynamic  synchronization  events,  e.g.  "call  me  when  you’re  ready",  and 
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dictates  that  any  coordinated  actions  among  team  participants  are  deterministically  pre¬ 
planned  and  scheduled  prior  to  plan  execution.  This  is  a  critical  shortfall  of  both 
controllers  because  the  MICA  vision  entails  dynamic  responsiveness  to  evolving 
guidance,  constraints  and  tactical  needs  and  emphasizes  cooperation  between 
heterogeneous  elements.  Current  controller  designs  are  limited  to  centralized  control  and 
rely  primarily  on  pre-planned  activities.  Furthermore,  neither  controller  accurately 
modeled  the  tactical  communication  environment  and  limitations  that  will  dramatically 
affect  real  life  employment. 

5.2.1.3  Are  multi-team  composition  and  activities  coordinated? 

The  construct  of  teams  within  the  current  controllers  appear  to  be  a  simple  partitioning  of 
assets  and  targets  arising  from  the  'decomposition'  of  the  challenge  problem  into  smaller 
sub-problems  that  can  be  planned  for  more  easily.  In  this  regard,  the  teams  do  not  appear 
to  be  cognizant  of  each  other,  and  any  intra-team  coordination  is  simply  the  product  of 
the  original  problem  decomposition. 


5.2. 1.4  Do  assets  employ  full  spectrum  of  weapons? 
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Figure  5.2.1.4-1  -  Iterativity-OSU  Planned  Weapon  Utilization 

The  controllers  did  plan  to  use  a  variety  of  weapons  as  illustrated  in  the  plan  sequences 
below,  Figure  5.2. 1.4-1,  but  the  favored  weapon  by  far  was  the  seeker  missile  which 
maximized  per  asset  weapon  carriage,  minimized  the  probability  of  collateral  damage, 
and  minimized  useless  expenditure  of  weapons  by  requiring  a  target  lock  prior  to  weapon 
release. 

The  first  plan  sequence  shows  a  small  weapon  UAV  launching  two  decoys  against  two 
separate  long  SAM  launchers,  flying  to  an  EW  radar  site,  and  launching  a  seeker  missile 
against  the  EW  radar.  The  second  plan  sequence  shows  another  small  weapon  UAV 
flying  to  a  medium  SAM  site,  dropping  a  GPS  bomb,  flying  to  another  medium  SAM  site 
and  dropping  another  GPS  bomb. 

5.2.1.5  Commanders  Guidance  and  the  Rules  of  Engagement 

The  Commander's  Guidance  and  Rules  of  Engagement  represent  objectives  that  must  be 
satisfied  and  planning  constraints  that  must  be  adhered  to  while  executing  a  military 
campaign.  These  planning  objectives  and  constraint  sets  were  initially  provided  to  the 
MICA  developers  as  a  PowerPoint  presentation  along  with  the  Challenge  Problem 
documentation.  They  have  since  been  translated  to  an  XML  representation  as  part  of  a 
"planning  language"  that  is  distributable  to  the  platforms  within  the  MICA  OEP  via  the 
communication  networks. 
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The  Draper  controller 
requires  that  the  MICA 
operator  translate 

Commander's  Guidance  and 
Rules  of  Engagement  into 
target  valuation  tables,  Figure 
5. 2. 1.5-1.  These  valuation 
tables  assign  relative 
awareness,  kill,  and  BDA 
values  to  the  target  types 
within  a  designated  area.  The 
burden  lies  with  the  operator 
to  select  values  that  produce 
the  desired  effect  and 
unfortunately  the  system 
provides  no  assistance  to  the 
operator  for  executing  this 
essential  task.  For  example, 
if  a  target  type  is  not  to  be 
prosecuted  outside  of  a  designated  'Kill  Zone',  the  operator  must  manually  set  its  relative 
kill  value  to  zero  in  all  other  areas.  Or  if  a  localized  target  set  must  be  destroyed  within  a 
given  time  frame,  its  associated  target  type  kill  value  may  initially  be  set  high  to  promote 
a  higher  probability  of  early  prosecution  task  assignment.  Then  as  the  objective  is 
approached,  the  kill  value  may  be  iteratively,  and  manually,  reduced.  It  is  these  target- 
task  values  that  are  then  used  in  the  planning  process  to  assign  asset-missions  to  targets 
by  maximizing  a  global  utility  function.  For  example,  discovering  and  destroying  high 
value  targets  add  more  to  the  global  utility  than  prosecuting  low  value  targets. 

The  Iterativity  VII  does 
allow  for  the  assignment  of 
one  of  four  predefined 
Rules  of  Engagement  (kill 
zone,  time  critical, 
hostilities,  or  no  strike)  to 
each  of  the  predefined 
mission  types  (SEAD, 

Interdiction,  or  CAS)  in 
each  of  the  Red  Areas, 

Figure  5. 2. 1.5 -2.  However, 
there  is  no  automated 
means  of  responding  to 
changes  in  Commander's 
Guidance  or  ROE.  With 
both  of  the  controllers,  it 
remains  the  MICA 

Operator's  responsibility  to  manually  reflect  any  guidance  and  ROE  changes. 


SSMSystem  red #2  SSM  System  Specification 


Interdiction  ROE: 

Potential  Interdiction  Targets 
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Figure  5.2.1.5-2  -  Iterativity  VII  ROE  Selection 


Figure  5.21.5-1  -  Draper  Target  Valuation  Table 
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Both  controllers  were  'unstable'  and  could  not  progress  sufficiently  against  the  challenge 
problem  to  enable  a  statistical  assessment  of  adherence  to  CG  and  ROE.  With  respect  to 
the  Draper  controller,  collateral  damage  against  white  occurred  even  in  the  limited 
experimentation  sample.  There  was  no  behavioral  demonstration,  for  either  controller, 
that  the  Rules  of  Engagement  were  actively  constraining  either  the  planning  process  or 
weapon  release.  Both  controllers  were  'unstable'  and  could  not  progress  sufficiently 
against  the  challenge  problem  to  enable  a  statistical  assessment  of  adherence  to  CG  and 
ROE.  With  respect  to  the  Draper  controller,  collateral  damage  against  white  occurred 
even  in  the  limited  experimentation  sample.  It  was  not  apparent  that  the  Rules  of 
Engagement  were  translated  to  logical  rules  or  targeting  constraints  that  were  assessed 
during  the  planning  process  or  prior  to  weapon  release. 

5.2.2  Team  Dynamics  and  Tactics 

Both  controllers  have  displayed  evidence  of  some  team  tactics:  1)  cooperative  jamming, 
where  one  asset  strikes  a  target  while  another  asset  jams;  2)  cooperative  bomb  damage 
assessment,  where  different  assets  are  allocated  for  weapon  release  and  subsequent  target 
imaging;  and  3)  pre-planned  mission  execution,  the  explicit  sequencing  of  actions 
performed  by  different  assets  toward  a  single  target.  Unfortunately,  effective  end-game 
team  (or  individual)  tactics  have  not  been  demonstrated. 

5.2.2.1  Jamming  Tactics 

The  OEP's  integrated  jammer  and  radar  models  predict  both  main- lobe  and  side-lobe 
jamming  effects  including  the  'additive'  contributions  of  multiple,  dispersed  emitters.  All 
SAM  tracking  radars  have  a  "guide-on-jammer"  capability  allowing  the  SAM  sites  to 
acquire,  track,  and  engage  any  platform  that  is  jamming  within  its  engagement  zone.  The 
simple  tactic  of  continuously  self  jamming  a  SAM  site  during  an  engagement  will  lead  to 
a  high  probability  self  destruction.  Effective  jamming  tactics  against  a  SAM  site 
therefore  requires  combinations  of  multiple  participants  having  more  complex  flight  plan 
geometries  inside  and  outside  of  the  effective  SAM  engagement  envelope  and  employing 
time-varying  techniques  that  take  advantage  of  SAM  lock  on  and  tracking  peculiarities. 


Figure  5.2.2.1-1  -  Draper  Cooperative  Jamming  Plan 

The  Draper  controller  performed  cooperative  stand-off  jamming  as  is  illustrated  in  the 
team  plan  scheduled  below,  Figure  5.2.2. 1-1.  In  this  plan,  two  members  of  the  team  are 
stand-off  jamming  while  a  small  combo  effectively  locates,  strikes,  and  performs  BDA 
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on  the  target.  These  tactics  were  effective  within  the  limitations  of  pre-planning  and  re¬ 
planning,  i.e.  all  mission  elements  were  pre-planned  and  in  many  cases  changes  in  the 
tactical  situation  initiated  replans  that  removed  needed  elements  from  these  cooperative 
scenarios.  In  re-planning  the  controller  failed  to  recognize  the  critical  inter-relationship 
between  cooperative  elements  and  would  re-allocate  one  or  more  jamming  contributors 
leaving  the  remaining  attack  asset(s)  vulnerable  to  the  red  defenses. 

The  Iterativity-OSU  Ultra  planner  made  extensive  use  of  jamming.  Unfortunately  there 
is  no  indication  of  cooperative  multi-platform  jamming  tactics.  The  Ultra  planner  simply 
attempts  to  intermittently  self  jam  targeted  SAM  sites.  An  example  of  this  unsuccessful 
self-jamming  technique  is  indicated  on  the  following  Ultra  plan  sequence  of  a  small 
combo  UAV  SEAD  mission  against  a  long  SAM  launcher,  Figure  5.2.2. 1-2.  The 
sequence  of  planned  actions  are:  Jam,  FlyTo,  Focate,  Identify,  Attack,  and  Assess.  The 
OSU-Hierarchical  planner  made  no  use  of  the  jamming. 
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See 
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<■>  ^ 

Figure  5.2.2.1-2  -  Iterativity-OSU  Self  Jam  Plan 

5.2. 2.2  Cooperative  Engagement  Tactics 

The  majority  of  blue  asset  types  provided  by  the  MICA  challenge  problem  do  not  have 
sufficient  native  capabilities  to  independently  perform  all  of  the  target  kill  chain 
activities,  Figure  5.2.2.2-1.  For  example,  the  weapon  platforms  do  not  have  sensors 
sufficient  to  precisely  locate  or  assess  a  target's  damage  state,  and  the  sensor  platforms 
have  no  weapons  with  which  to  strike  a  target.  Even  the  combo  platforms  are  limited  in 
their  capability  to  operate  alone.  Thus  cooperative  tactics  for  target  engagement  are 
imperative. 


Find  )  Fix  7  Track /Target)  Engage)  Assess 


Figure  5.2.2.2-1  -  Target  Kill  Chain 


All  of  the  controllers  planned  for  cooperative  target  prosecution  by  sharing  the  various 
activities  of  the  kill  chain  among  the  members  of  a  heterogeneous  strike  team.  Task 
dependencies  are  strongly  indicated  with  the  Iterativity-OSU  Hierarchical  and  Ultra 
planners  by  graphically  representing  activity  precedence  and  concurrence  as  connecting 
lines  in  the  partially  ordered  plan  sequences.  The  plan  snippet  below,  Figure  5.2. 2.2-2 
shows  concurrent  decoy  and  seeker  missile  launches  by  a  weapon  platform  against  a 
large  SAM  tracking  radar  and  the  enforced  precedence  ordering  of  the  following  imaging 
task. 
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Figure  5.2.22-2  -  Iterativity-OSU  Inter-plan  Task  Dependencies 


Planned  coordination  of  team  activities  in  the  Draper  controller  can  be  illustrated  with  the 
"Team  Synchronization  Matrix"  shown  in  Figure  5.2. 2.2-3.  This  plan  segment  shows  the 
distribution  of  activities  across  a  heterogeneous  team  of  UAVs  engaging  a  long  SAM 


Team  Synchronization  Matrix 

h 

X 

Zoom  In  Full 

Zoom  Tn  1  Zoom  Out  Fit  to  Window  Prepend  Histonr 

* 

A 

Team  5-14 

1  II 

i  i  i  i  i  1 1  * 

i*i  N 

III 

_ 1 

A 

Team  5  -  22 

i*n 

1  1  1  1  1  1  1  9 

1  #  1  9  IT 

III 

X 

Team  5  -  25 

1  II 

1  1  1  1  1  1  1  9 

1  *  1  II 

X 

Team  5  -  33 

1  II 

1  1  1  1  1  1  1  9 

1*1  II 

III 

| 

X 

Team  5  -  45 

1  II 

llllll  Hi 

m 

III 

J 

X 

Team  5  -  51 

1  IL 

Mini  i  ^ 

1  9  1  II 

III 

“1 

.egend 


|  Strike  Q  Sense  Q  Area  Search 
Q  Refuel  Q  Jam  Q  Take  off 


|  Load  Weapons  [~]  Deploy  Decoy 
|  Land  Q  Fly  to  WP 


Figure  5.2.2.2-3  -  Draper  Synchronization  Matrix 

tracking  radar.  The  initial  target  location  and  identification  tasks  and  the  final  bomb 
damage  assessment  has  been  allocated  to 
one  of  the  two  large  sensor  platforms. 

The  weapon  release  has  been  assigned  to 
the  weapon  platform  having  the  smallest 
signature.  And  all  other  team  members 
perform  stand-off  jamming  against  the 
long  SAM  tracking  radar. 

5.2.2.3  Multilateration  Teams 

Multilateration,  Figure  5. 2.2. 3-1,  is  the 
most  effective  tactic  for  locating  and 
identifying  emitting  targets. 

Additionally,  multilateration  is 
necessarily  a  team  tactic  requiring  the 
connection  of  team  participants  to  a 

shared  data  network.  Neither  of  the  Figure  5.2.2.3-1  -  Multilateration 

controllers  fundamentally  planned  for 
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use  of  multilateration  teams. 


Changes  to  the  passive  ranging  model  were  made  in  OEP  version  1 .2  formally  available 
in  May  2003.  The  original  single  ship  passive  ranging  model  was  replaced  with  a  multi¬ 
ship  model  requiring  team  formation  for  precise  location  and  identification  of  RF 
emitters.  For  ESM  sensors  to  generate  tracks,  they  must  first  be  connected  to  'networked' 
multilateration  components,  rather  than  directly  to  the  tracklists  in  the  platforms.  A 
multilateration  team  is  then  established  by  three  or  more  platforms  connecting  to  a 
common  network  and  sharing  ESM  sensor  reports.  The  accuracy  of  resultant  reports  can 
be  extremely  high  if  proper  geometry  between  platforms  and  the  observed  emitter  is 
maintained.  Because  of  this  geometric  sensitivity,  multilateration  teams  should  be 
constituted  and  their  geometric  spacing  planned  as  part  of  an  overall  system  solution. 

During  the  initial  experimentation,  it  was  discovered  that  the  Draper  controller  made  no 
use  of  multilateration  for  target  location  and  identification.  Multilateration  teams  were 
subsequently  added,  though  this  was  done  without  a  logical  "network  connection  action". 
The  resultant  ad-hoc  multilateration  teams  are  not  a  driver  in  the  development  of  mission 
plans,  but  are  created  after  other  tasks  have  been  allocated  and  the  routs  have  been 
planned.  Multilateration  team  geometry  thus  arises  accidentally  from  other  path  planning 
considerations.  Frequently,  multilateration  team  participants  have  minimal  geometric 
separation  relative  to  their  targets.  Occasionally,  one  member  of  a  three  ship 
multilateration  team  would  still  be  located  at  the  blue  base  on  the  ground,  and  therefore 
would  not  be  able  to  generate  multilateration  reports.  In  any  case,  these  conditions 
essentially  made  multilateration  ineffective. 

It  was  also  determined  during  experimentation  that  neither  Iterativity-OSU  planner  used 
multilateration.  No  'reasonable'  network-connection  activity  was  defined  and  no 
platforms  were  connected  to  the  multilateration  networks. 

The  simple  addition  of  multilateration  would  have  greatly  improved  the  information 
gathering  performance  of  both  the  Draper  and  Iterativity-OSU  controllers.  By  having  the 
UAVs  in  a  local  area  publish  their  locations,  they  could  then  determine  locally  optimal 
team  geometries  and  form  ad-hoc  multilateration  teams.  This  would  have  allowed  for 
accurate  discovery,  localization,  and  identification  of  every  emitting  platform  in  a 
covered  region. 

5.2.2.4  Planning  Triggers 

Planning  triggers  are  critical  elements  of  MICA  controller  design.  A  balance  must  be 
struck  between  committing  to  existing  plans,  in  order  to  complete  tactical  objectives  and 
replanning  to  achieve  a  more  optimal  overall  solution.  Effective  controller  design 
supports  automated  local  and  global  triggers,  subject  to  operator  concurrence,  coupled 
with  the  cap  ability  for  the  operator  to  initiate  similar  partial  or  global  replans.  Each  of 
the  evaluated  controllers  offer  elements  of  the  desired  capability  but  neither  affords  all 
desired  features. 
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The  Iterativity-OSU  controller  allow  user  selection  of  planning  triggers  including  New 
Target  discovery,  Asset 
Loss,  and  Time  Based 
replanning,  Figure  5.2.2A- 
1 .  Additionally  the 
Variable  Initiative  Interface 
allows  for  MICA  operator 
initiation  of  a  team  level  re¬ 
plan,  see  below.  However, 
the  operator  is  not  afforded 
the  opportunity  to  concur 
with  the  replan  nor  is  the 
operator  able  to  initiate  a 
global  replan. 


The  primary  planning 
trigger  for  the  Draper 
controller  appears  to  be 
cumulative  changes  to  the 
Information  Model,  i.e. 
discovery  of  new  targets, 
movement  of  known 
targets,  or  changes  to  the 
target  probability 

distributions.  Additionally 
the  Draper  controller  re¬ 
plans  at  both  the  TCT 
(Team  Composition  and  Tasking)  and  TDT  (Team  Dynamics  and  Tactics)  levels  on  a 
predefined  time  interval.  TCT  globally  re-plans  every  hour  and  the  TDT  re-plans  on  a 
shorter  time  interval.  It  was  not  evident  that  re-planning  is  initiated  by  asset  loss,  and 
there  is  no  user  interface  to  dynamically  influence  re-plan  criteria  or  to  initiate  a  replan. 
Below  is  a  'message  log'  illustrating  initiation  of  a  top  level  TCT  re-plan  request  due  to 
cumulative  changes  to  the  Information  Model  which  have  apparently  altered  the 
calculated  utility  of  the  current  plans. 


FligntGroup  Team  1  Specification 
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Figure  5.2.2.4-1  -  Iterativity-OSU  Trigger  Selection 
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The  net  result  is  that 
the  Draper  plans  are 
reset  excessively, 
inhibiting  any 

significant  progress 
toward  achieving  the 
campaign  level  goals. 

Assets  initially 

allocated  to  one  team 
will  often  get  re¬ 
assigned  to  new  a 
team  or  sent  back  to 
base  without 

accomplishing  any 
assigned  tasks. 

Additionally,  many  of 
the  new  'teams'  are 

composed  of  very  few  assets.  A  better  balance  needs  to  be  achieved  on  commitment  to 
existing  plans  and  the  extent  and  frequency  initiating  re -plans.  Allowing  for 
customization  and  control  of  planning  triggers  by  a  MICA  Operator  would  be  a  powerful 

tool  to  enable  better  control  of  the 
replan  cycle  in  real-time. 


Figure  5.2.3-1  -  Iterativity-OSU  “Gradient  Descent”  Planner 


5.2.3  Cooperative  Path 
Planning 

Both  controllers  exhibited 
evidence  of  coordinated  team 
tactics  and  planned  asset  flight 
paths  in  response  to  either  team 
assignment  or  task  allocation. 

Both  controllers  currently  plan  and 
fly  in  a  horizontal  plane,  i.e.  the 
path  planners  are  two-dimensional. 
The  Draper  controller  flies  at 
4000m  while  the  OSU-Iterativity 
controller  files  at  5000m..  The 
planned  flight  paths  of  the  Draper 
controller  deviate  from  the  plane 
but  only  for  an  asset  to  takeoff  and 
land. 


Figure  5.2.3-2  -  Draper  Global  Search  Planner 


This  implies  that  neither  of  the 
controllers  are  rigorously  applying 
or  taking  advantage  of  the  three- 
dimensional  aircraft  signatures  in 
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their  path  planning.  Signature  management,  when  engaging  a  long  SAM,  can  greatly 
increase  the  probability  of  UAV  survival. 

The  OSU-Iterativity  controller  has  implemented  a  potential  based  "gradient  descent" 
local  path  planner,  Figure  5.2.3- 1.  The  potential  is  based  upon  known  threats.  Escape 
from  local  potential  minima  is  handled  by  uniformly  lowering  the  calculated  threat 
potential.  While  this  will  guarantee  UAV  progress  toward  its  goal,  the  path  planner  is  not 
likely  to  find  the  lowest  risk  path.  The  benefit  is  that  the  planner  will  inherently  react  to 
discovered  threats  by  adding  them  to  the  composite  threat  potential. 

Draper  seems  to  have  implemented  an  entirely  global,  graph  search  based  path  planner, 
Figure  5. 2. 3-2.  Additionally  Draper  seems  to  have  tightly  coupled  the  task  allocation  and 
path  planning  in  their  TDT/CPP  planner.  A  significant  drawback  of  this  approach  is  that 
they  have  not  demonstrated  a  capability  to  perturb  a  planned  path  segment  to  avoid  a 
discovered  threat  without  initiating  some  level  of  TDT/CPP  re-planning.  A  25  Km 
rectilinear  grid  is  used  for  global  path  planning.  Initially,  paths  appeared  to  be 
constrained  to  the  grid,  deviating  only  when  necessary  to  engage  (sensor  or  weapon)  a 
target.  With  later  releases,  the  planned  paths  have  deviated  more  significantly  from  the 
course  grid.  This  allows  for  more  survivable  routs  when  traversing  a  complex  threat 
environment.  The  Draper  results  would  have  been  even  less  promising  if  they  had  not 
taken  advantage  of  “perfect  IPB,  discussed  below. 

Because  Iterativity-OSU  enables 
localized  replanning,  they  are  able 
to  account  for  discovered  threats 
without  abandoning  a  plan  in 
progress.  As  would  be  expected, 
these  localized  reactions  appear  to 
improve  controller  performance. 
In  contrast,  the  Draper  controller 
does  not  support  localized 
replanning,  is  unable  to  make  these 
surgical  adjustments,  and  thus 
suffer  additional  attrition  and  loss 
of  progress  due  to  excessive  global 
replanning. 


5.2.4  Uncertainty  Management 

The  MICA  IPB  (Intelligence 
Preparation  of  the  Battlefield)  was 
designed  to  realistically  reflect  the 
uncertainty  of  information 
available  to  the  planning  process 
prior  to  initiation  of  a  campaign. 

Figure  5.2.41-1 -Iterativity  IPB  Entry  The  ability  of  the  controllers  to 

manage  varying  amounts  of 
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uncertainty  was  to  be  evaluated  primarily  by  varying  the  completeness  and  accuracy  of 
the  provided  IPB.  Proper  use  of  this  IPB  was  deemed  essential  to  controller  performance 
as  not  all  targets  and  threats  would  be  precisely  known  prior  to  beginning  an  operation. 
Unfortunately,  the  limited  performance  of  the  controllers  against  the  MICA  challenge 
problem,  even  with  a  "perfect  IPB",  limited  the  value  of  a  detailed  assessment. 

5.2.4.1  Initial  Preparation  of  the  Battlefield 

The  MICA  Challenge  Problem  and  OEP  provided  both  qualitative  and  quantitative  IPB 
content.  Quantitative  IPB  information  was  accessible  to  the  controllers  via  initial  data  in 
the  various  platform  tracklists  available  upon  scenario  startup,  and  all  controllers  have 
made  use  of  this.  Qualitative  IPB  information  was  initially  available  to  the  MICA  "UAV 
Team"  Operators  as  a  PowerPoint  presentation,  much  as  it  is  handled  in  the  field  today. 
In  response  to  a  request  from  Draper  during  early  experimentation,  a  preliminary  XML 
representation  of  this  qualitative  information  has  been  developed  and  is  accessible  via  the 
GetProperty("IPB")  API  call. 

Prior  to  provision  of  the  XML  representation,  Draper  had  been  generating  a  "perfect 
qualitative  IPB"  which  was  used  to  initialize  their  Information  Model,  discussed  below. 
It  was  discovered  during  experimentation  that  the  Draper  controller  was  consuming  the 
Challenge  Problem  XML  and  storing  red  and  white  platform  quantity  and  type  as  a 
probability  mass  function  across  the  scenario  area.  This  PMF,  based  on  truth  data,  was 
then  used  to  deterministically  plan  flight  paths  around  known  but  as  of  yet  'undiscovered' 
threats,  and  to  initiate  sensor  missions  to  establish  tracks  for  targets  that  were  known  to 
exist,  but  had  not  yet  been  discovered. 
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Figure  5.2.4.2-1  -  Draper  Information  Model 


Alternatively,  the  Iterativity  VII 
provides  an  intuitive  USI  for  input 
of  the  qualitative  IPB  data 
provided  in  the  PowerPoint  File, 
Figure  5.2.4. 1-1.  A  set  of  potential 
target  types,  associated  quantities, 
and  Rules  of  Engagement  for  each 
geographic  area  is  presented  to  the 
MICA  Operator  for  selection  and 
data  input.  Corresponding 
symbols  are  then  placed  in  each  of 
the  geographic  areas  allowing  for 
subjective  placement  of  the  targets 
and  assignment  of  their  location 
errors. 

5.2.4.2  Draper  Information 
Model  or  Probability 
Mass  Function 

Draper  has  created  a  target 
Information  Model  that  maintains 
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estimations  of  target  type, 
location,  and  state  across 
the  geographic  region  of 
interest.  Figure  5.2.4.2-1. 
In  principle,  this  two 
dimensional  target 

distribution,  or  probability 
mass  function,  is  initialized 
from  both  the  Intelligence 
Preparation  of  the 
Battlefield  and  the  targets 
known  a  priori,  i.e.  the 
Initial  Track  List.  OEP 
Track  Events  are  then  used 
to  update  and  maintain  this 
model. 

This  information  model 
maintains  a  representation 
of  both  uncertain  targets, 
targets  that  are  expected  to 
be  found  in  a  general  area 
but  for  which  a  track  does 
not  yet  exist,  and  'certain 
targets'  for  which  do  tracks 
exist  and  are  being 
maintained. 


Figure  5.2A.2-2  -  Draper  ISR  Mission  Plan 


It  is  this  naturally 

complementary  representation  of  target  information  that  is  then  used  to  support  task 
selection,  allocation  and  path  planning.  For  example,  the  expectation  of  several  uncertain 
targets  within  in  a  local  area  may  be  used  to  initiate  and  plan  ISR  missions,  Figure 
6. 2.4. 2-2.  Or  the  existence  of  'certain  tracks'  of  known  type  and  state  may  be  used  to 
initiate  strike  missions  and  the  existence  tracks  of  uncertain  type  or  state  may  be  used  to 
initiate  ID  or  BDA  missions.  Also  the  presence  of  both  certain  and  expected  threats  may 
be  used  for  path  planning  and  threat  avoidance. 


Figure  5.2.4.2-2  represents  an  ISR  mission  planned  accordingly.  As  can  be  seen,  the  path 
has  been  planned  to  avoid  the  known  threats,  and  the  search  areas  □  correspond  to  PMF 
cells  with  large  target  expectation  highlighted  above. 


In  light  of  the  Information  Model  structure,  and  its  initialization  by  parsing  of  the 
Challenge  Problem  XML,  it  can  be  understood  why  the  Draper  controller  does  not 
respond  to  threat  warning  reports  and  only  rarely  retrieves  OEP  Events.  The  various 
planners  reason  upon  the  Information  Model,  and  are  triggered  by  changes  to  the 
modeled  state.  First,  since  any  change  to  the  represented  state  might  be  significant,  the 
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controller  re-plans  with  every  state  change,  no  matter  how  in-significant,  i.e.  a  small 
change  in  target  location  or  perhaps  a  change  in  target  classification  probability  triggers  a 
full  replan.  Second,  even  though  the  Information  Model  can  represent  a  target  as  a  SAM, 
including  its  location  and  damage  state,  there  does  not  appear  to  be  a  means  of 
representing  that  it  is  actively  engaging  a  blue  UAV.  Because  of  this  limitation,  the 
planners  cannot  reason  upon  this  un-represented  state  and  determine  that  the  engaged 
UAV  needs  to  take  immediate  evasive  maneuvers.  With  the  Information  Model,  Draper 
has  constructed  an  elegant  vehicle  for  representing  the  uncertain  nature  of  knowledge  of 
the  battlefield.  Unfortunately,,  because  of  the  incomplete  state  representation,  or  perhaps 
because  of  the  way  it  is  integrated  with  the  planners,  it  is  not  well  suited  for  driving  local 
reactive  planning  such  as  the  exemplified  evasive-maneuver.  These  problems  may  be 
mitigated  by  expanding  the  modeled  state  space  and  by  'tuning'  the  algorithm  used  to 
initiate  team  level  re-planning. 

5.2.5  Variable  Initiative 

The  Variable  Initiative  Interface  is  intended  to  provide  the  MICA  operator  information 
sufficient  to  assess  both  the  global  situation  and  the  appropriateness  and  soundness  of  the 
proposed  team  plans.  Additionally,  it  is  intended  to  allow  the  operator  to  interact  with 
the  decision  process,  and  in  the  extreme,  control  the  various  MICA  assets. 


Figure  5..2.5.1-1  -  Iterativity  VII  Display 
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5.2.5.1  The  Iterativity  Variable  Initiative  Interface 

There  are  at  least  two  outstanding  features  of  the  Iterativity  VII,  Figure  5.2. 5. 1-1.  First, 
the  graphical  representation  of  the  partially  ordered  plan  sequence  with  "activity  tiles" 
proved  to  be  an  effective  approach  to  intuitively  conveying  what  activities  are  being 
planned  and  when  (with  respect  to  order)  they  will  occur.  Time  is  not  explicitly 
represented,  but  this  may  be  a  strength  since  the  critical  element  in  planning  is  the 
sequence  of  events  rather  than  maintaining  a  rigid  schedule. 

A  further  improvement  would  allow  the  MICA  Operator  to  use  this  "activity  tile"  plan 
representation  as  means  for  interacting  with  the  planning  process.  The  operator  might 
explicitly  assign  a  platform  to  a  specific  target,  or  restrict  the  weapon  type  to  be  used 
against  a  target.  Or  perhaps  the  operator  could  enter  coordination  events  to  impose  a 
sequencing  on  the  Order  of  Battle.  Additionally,  it  would  be  beneficial  to  have  a 
corresponding  "target  centric"  representation  of  this  plan  interaction  interface. 

The  Iterativity  VII  allowed  for  intuitive  input  of  additional  information  to  be  used  by  the 
planning  process.  As  discussed  previously,  the  Iterativity  VII  allowed  for  intuitive  input 
of  qualitative  battlefield  intelligence,  i.e.  approximate  quantities  of  targets  in  approximate 
locations.  Additionally  the  VII  allowed  for  intuitive  assignment  of  Rules  of  Engagement 
to  pre-defined  target  clusters  as  well  as  ranking  of  the  various  Commander's  Guidance 
Objectives  to  be  used  in  the  planning  process.  What  would  have  made  this  even  better 
would  be  to  be  able  to  define  "new"  Rules  of  Engagement,  Commanders  Guidance,  and 
target  clusters. 


Figure  5.2.5.2-1  -  Draper-Charles  River  VII  Display 


32 


Two  drawbacks  of  the  Iterativity  VII  were  the  lack  of  visualization  of  planned  paths,  and 
a  lack  of  means  to  edit  a  generated  plan  sequence  or  directly  interact  with  the  planning 
processes,  though  this  was  a  shortcoming  of  all  of  the  controllers. 

5.2.5.2  The  Draper  -  Charles  River  Variable  Initiative  Interface 

The  Draper  VII  was  separated  into  a  planning  and  control  MICA  Information  Display, 
and  a  situational  awareness  Human-System  Interface  visualization  display,  Figures 
5.2. 5.2-1  and  5.2. 5.2-2.  This  separation  naturally  reflects  the  main  purposes  of  the  VII, 
the  monitor  and  control  of  MICA  teams. 

The  MICA  Information  Display,  Figure  5. 2. 5. 2-1,  allowed  for  both  asset  centric  and 
target  centric  viewing  of  generated  plans,  but  as  with  the  Iterativity-OSU  controller,  there 
were  no  means  of  direct  interaction  during  plan  formation  nor  of  subsequently  editing  the 
generated  plans.  Interaction  with  the  planning  process  was  largely  limited  to  adjusting 


Figure  5.2.5.2-2  -  Draper-Charles  River  HSI 
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target  table  valuations  prior  to  plan  auto-generation,  and  the  subsequent  acceptance  or 
rejection  of  the  plan  in  its  entirety. 

Furthermore,  there  were  no  means  for  entering  additional  information  such  as  qualitative 
battlefield  intelligence,  new  strategic  objectives,  or  changes  to  the  rules  of  engagement  - 
other  than  tuning  of  the  target  valuation  tables.  This  approach  appears  to  be  more 
algorithm  oriented  than  operator  oriented.  For  example,  this  forces  the  MICA  Operator 
to  discover  how  to  translate  a  strategic  objective  into  a  specific  target  value  distribution, 
rather  than  simply  entering  a  symbolic  (logic)  representation  of  the  strategic  objective. 

The  Draper  HSI  display  provides  several  intuitive  plan  visualization  tools  such  as  a 
planar  graph  representation  of  planned  paths  with  selectable  activity  symbols  such  as 
search,  strike,  and  sense.  The  HSI  could  be  decluttered,  if  beneficial,  by  displaying  only 
selected  assets  and  teams.  Additionally,  the  HSI  provides  a  "Plan  Animation"  feature 
allowing  a  "preview"  of  the  current  plans  as  they  are  anticipated  to  play  out. 

5.2.5.3  Variable  Initiative  Interface  Summary 

As  discussed  above,  no  VII  actually  allowed  the  MICA  Operator  to  directly  interact  with 
the  planning  process.  No  VII  allowed  the  operator  to  edit  a  subsequent  plan,  either  at  the 
team  composition  level,  or  at  the  task  allocation  level.  Providing  these  capabilities,  as 
well  as  a  mean  to  visualize  and  implement  higher  level  strategies,  would  greatly  enhance 
the  utility  of  the  MICA  Variable  Initiative  Interface. 

5.3  Challenge  Problem  Tractability 

Several  researchers  have  contended  that  their  lack  of  progress  against  the  MICA 
Challenge  Problem  is  reflective  of  an  impossible  degree  of  difficulty  in  terms  of  IADS 
capability  vs.  blue  assets.  This  issue  can  be  partially  addressed  through  comparison  with 
the  Boeing  BlueController,  whose  quantitative  performance  exceeded  that  of  the 
heavyweight  MICA  Controllers  in  terms  of  the  kill  ratio  between  red  and  blue  assets,  but 
whose  algorithms  were  lightweight  in  comparison. 

The  Boeing  Blue  Controller  evolved  over  the  final  18  months  of  the  MICA  program.  It 
was  initially  developed  as  a  tool  for  robustly  debugging  the  new  models  and  APIs  that 
were  added  to  the  OEP.  Its  capabilities  grew  under  the  influence  of  other  programs 
(AFRL  REAC)  and  research  efforts,  and  under  the  MICA  need  to  stimulate  and  challenge 
the  IADS  Red  Controller.  Perhaps  the  most  notable  distinction  of  the  Blue  Controller 
from  the  heavyweight  MICA  Controllers,  other  than  algorithm  complexity,  was  that  the 
Blue  Controller  consumes  a  larger  subset  of  OEP  events,  most  notably  the  Threat 
Warning  Event.  This,  and  a  short  event  handling  cycle  of  5  seconds,  allowed  the  Blue 
Controller  to  rapidly  perceive  and  react  to  the  changing  adversarial  environment,  and 
particularly  to  engagements  by  red  SAM  threats. 

Briefly,  the  Blue  Controller  was  built  around  an  auction  based  task  allocation  protocol 
and  a  centralized  KnowledgeObject,  Figure  5.3-1.  Auction  based  task  allocation  allowed 
algorithms  to  be  located  where  requisite  data  was  naturally  resident,  thus  minimizing 
communication  requirements  among  the  various  platforms.  Examples  of  distributed 


34 


algorithms  are  Cost  and  Merit  calculation  used  for  Utility  calculation  and  the  reactive 
threat  avoidance  algorithm.  In  contrast,  the  KnowledgeObject  defines  the  "universe  of 
discourse"  and  allows  specialized  algorithms  to  operate  upon  a  shared  representation  of 
the  battlespace.  Examples  of  specialized  algorithms  would  be  the  application  of  ROE 
constraints  to  target  selection,  or  the  greedy  task-asset  selection  algorithm,  which 
combines  the  cost  and 
merit  reported  by  each 
asset  into  a  communal 
utility,  which  is  then 
maximized  to  determine 
the  optimal  asset  for  each 
task  assignment. 

Even  with  these  relatively 
lightweight  algorithms, 

BlueController 
performance  often 

exceeded  that  of  the 
evaluated  MICA 

Controllers.  While  these 
would  lose  most  if  not  all 
of  their  assets  while 
destroying  2-3  SAM  sites 
(a  ratio  of  1  red  asset 
destroyed  for  every  12  to 
18  blue  assets  lost),  the 
light-weight  BlueController  tended  to  destroy  3  red  assets  for  every  5  blue  assets  lost, 
demonstrating  consistent  progress  against  the  full  MICA  Challenge  Problem.  More 
importantly  because  the  BlueController  was  focused  to  the  primary  Blue  objective  of 
eliminating  the  IADS  by  destroying  the  redundant  red  C2  facilities,  it  was  often  able  to 
accomplish  this  task.  Once  the  IADS  is  eliminated,  the  BlueController  consistently 
demonstrates  a  significant  tactical  advantage  over  the  red  SAMs  that  must  operate 
individually.  Neither  of  the  heavyweight  controllers  was  ever  able  to  destroy  either  of 
the  C2  facilities.  Furthermore,  the  BlueController  achieved  these  results  without 
employing  jamming.  The  Challenge  Problem  is  designed  to  require  cooperative  jamming 
for  IADS  penetration  with  minimal  blue  asset  loss.  We  are  confident  that  the 
BlueController  could  successfully  eliminate  the  IADS,  by  destroying  the  C2  facilities, 
while  suffering  minimal  loss  of  blue  assets  once  cooperative  jamming  techniques  are 
integrated  into  the  controller.  Unfortunately  this  planned  BlueController  enhancement 
could  not  be  completed  during  the  shortened  research  period. 

Regarding  Challenge  Problem  tractability,  we  would  also  reference  the  UCSB  research 
results.  UCSB  developed  a  controller  that  focused  on  managing  blue  vehicle  orientation 
relative  to  red  defenses  coupled  with  cooperative  jamming.  Their  results  confirmed  our 
Challenge  Problem  design  expectations  and  demonstrated  a  significant  tactical  advantage 
for  blue  over  red.  Based  on  our  experience  with  the  BlueController  and  the  UCSB 
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demonstrated  results,  we  are  confident  that  the  Challenge  Problem  is  tractable  and  is 
representative  of  real  world  challenges. 


6  Conclusions 

Although  much  was  accomplished  significant  research  is  still  needed  due  to  the 
premature  termination  of  the  program.  OEP  research  needing  to  be  completed  relates  to 
completion  of  the  planned  experimentation.  This  involves  more  than  execution  of  the 
experiments.  There  are  additional  OEP  capabilities  required  to  execute  each  phase  of 
experimentation  as  well  as  additional  controller  capabilities.  Most  importantly,  early 
experimentation  has  confirmed  the  extreme  value  of  the  integration  process  in  improving 
controllers  and  focusing  them  ob  tactical  utility.  The  remainder  of  this  section  will 
address  requisite  activities  and  capabilities  associated  with  each  experimentation 
hypothesis. 

6.1  HI  Experimentation 

The  original  objective  of  HI  testing  was  to  quantify  the  benefit  of  MICA  research  by 
comparing  execution  of  the  MICA  Challenge  Problem  by  USAF  domain  experts  using 
currently  available  technology  against  the  performance  of  MICA  controllers.  Towards 
this  end  a  top-level  battle  plan  for  the  MICA  CP  was  developed  by  the  133rd  Air  National 
Guard  (ANG).  The  original  intent  was  to  eventually  involve  representatives  of  the  133rd 
ANG  in  execution  of  the  full  challenge  problem  scenario.  This  execution  would  have 
provided  a  benchmark  for  absolute  evaluation  of  controller  performance.  This  benchmark 
will  be  required  if  any  absolute  measure  of  controller  performance  is  to  be  derived. 

It  is  worth  mentioning  that  the  HI  Experimentation  that  was  performed  has  proven 
extremely  valuable  during  our  limited  H2  and  H3  testing  discussed  above.  The  HI 
experimentation  has  highlighted  the  absence  of  a  top  level  battle  strategy  in  the 
controllers  evaluated  and  has  led  us  to  the  conclusion  that  future  automated  controllers 
need  to  incorporate  such  a  strategy.  Clearly  automated  controllers  need  to  integrate  the 
lessons  learned  through  years  of  battle  management  experience  if  they  are  to  be 
successful.  This  experience  dictates  that  a  sound  overall  battle  plan  and  strategy  are  the 
first  element  of  a  successful  campaign. 

We  believe  that  complete  execution  of  the  HI  Experimentation  would  yield  many 
additional  lessons  that  should  be  integrated  into  a  successful  controller  design.  As  the  old 
adage  says,  “No  battle  plan  last  beyond  the  first  shot.”  It  would  be  enlightening  to  see  if 
automated  controllers  are  as  successful  at  adjusting  as  are  competent  domain  experts. 
Expansion  of  the  OEP  visualization  capability  would  greatly  assist  in  execution  of  this 
expanded  Hi  Experimentation.  Visualization  is  essential  to  integrating  additional  human 
in  the  loop  control  into  MICA  execution  and  would  facilitate  qualitative  evaluation  of 
both  human  and  automated  controller  performance. 

6.2  H2  Experimentation 

Completion  of  H2  experimentation  would  greatly  enhance  the  overall  capability  of  MICA 
controllers.  Significant  improvements  in  the  Draper  controller  were  effected  in  the  short 
time  available.  Much  greater  progress  would  have  been  made  if  time  were  available  to 
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complete  testing  for  all  factors  and  most  importantly  for  Draper  to  respond  to 
experimentation  feedback  by  incorporating  additional  features  and  logic.  Unfortunately 
Draper  was  the  only  research  organization  that  afforded  Boeing  the  opportunity  to  both 
perform  experimentation  and  to  engage  in  a  dialogue  with  controller  designers.  It  is  clear 
that  if  time  and  resources  had  permitted,  continued  H2  experimentation  would  have 
benefited  each  controller  and  would  have  resulted  in  far  more  tactical  utility  from  the 
MICA  research. 

An  OEP  capability  that  would  have  greatly  enhanced  this  expanded  H2  Experimentation 
if  the  Red  Cell.  The  envisioned  Red  Cell  capability  would  have  added  to  the  current 
automated  red  IADS  by  adding  capability  for  closed  loop  control  of  red  ground  assets  and 
by  introducing  the  capability  to  integrate  human  decision  making  into  the  red  control. 
This  type  of  experimentation  would  be  of  great  value  as  the  automated  blue  controllers 
began  to  mature.  It  would  have  generated  additional  avenues  of  research  for  the  blue  side 
and  would  have  resulted  in  a  much  more  robust  and  useful  set  of  MICA  research  artifacts. 
Expanded  visualization  capability  would  have  been  critical  to  the  human  in  the  loop  red 
cell  capability  because  it  would  enable  the  red  commander  to  effectively  employ  his 
assets. 

6.3  H3  Experimentation 

Our  limited  experimentation  with  the  Draper  and  Steinmetz/OSU  controller  confirmed 
the  importance  of  the  variable  element  to  MICA.  We  were  strongly  impressed  with  the 
human  interface  of  the  Steinmetz/OSU  controller  and  believe  that  additional  innovation 
in  this  area  is  critical.  In  particular  giving  tools  to  the  commander  to  monitor  and  adjust 
the  battle  plan  as  well  as  the  tempo  and  extent  of  replanning  appears  to  be  powerful. 
Again,  expansion  of  the  OEP  visualization  capability  would  aid  in  this  area. 

6.4  H4  Experimentation 

This  area  is  critical  if  MICA  controllers  are  to  gain  credibility  as  transitionable  products. 
Little  beyond  planning  has  been  accomplished  in  this  area.  An  additional  capability 
required  to  execute  H4  Experimentation  is  the  interface  between  the  OEP  and  the  vehicle 
controllers.  Improved  visualization  would  greatly  enhance  the  impact  and  effectiveness 
of  H4  Experimentation. 

6.5  Summary  of  Additional  Research 

In  addition  to  complete  execution  of  the  MICA  Experimentation  Plan,  three  additional 
OEP  capabilities  are  required  to  complete  the  research.  Additional  effort  in  the 
visualization  interface  would  benefit  all  phases  of  experimentation.  The  Red  Cell 
capability  would  enhance  the  quality  and  extent  of  H2  Experimentation  and  would  yield 
more  robust  and  tactically  significant  controllers.  An  OEP  to  Mission  Control  Station 
interface  would  enable  H4  Experimentation  and  would  facilitate  transition  of  MICA 
research  artifacts  to  operational  employment. 
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Appendix  A  -  Planned  Experimentation 


At  the  time  that  MICA  Program  funding  was  terminated,  Boeing  was  nearly  finished 
planning  the  Experimentation  phase  of  the  program.  The  following  sections  detail  the 
planned  experimentation.  Results  of  the  limited  experimentation  that  Boeing  was  able  to 
accomplish  are  discussed  in  section  5. 

A.l  Statement  of  MICA  Experimentation  Objectives 

The  objective  of  MICA  experimentation  is  to  assess  MICA  research  products  to:  1) 
Ensure  that  they  represent  a  step  increase  in  operational  capability  over  currently  planned 
approaches  to  battle  management,  2)  Determine  the  extent  to  which  they  respond  to  each 
of  the  five  research  areas,  and  their  robustness  in  adapting  to  a  myriad  of  tactical 
situations,  and  3)  Ensure  that  they  are  transitionable  to  currently  planned  tactical  and  C2 
Systems.  The  sequential  nature  of  this  experimentation  is  designed  to  ensure  efficient 
expenditure  of  research  and  experimentation  resources.  The  latter  phases  of 
experimentation  will  be  performed  only  if  the  initial  phase  substantiates  that  MICA 
research  artifacts  demonstrate  a  significant  increase  in  operational  capability  over  current 
planned  tools  and  processes.  The  second  phase  will  be  applied  only  to  the  extent  that 
each  controller  shows  promise  in  a  specific  area.  To  this  end  controllers  will  be  pre¬ 
screened  to  ascertain  the  extent  to  which  their  design  accommodates  specific  areas  of 
experimentation.  The  final  phase  of  experimentation,  which  involves  a  mixed 
environment  with  hardware  operating  in  conjunction  with  a  synthetic  environment,  will 
only  be  applied  to  those  controllers  that  demonstrate  the  most  operational  utility  and 
robustness  during  the  first  two  phases. 

A.2  Experimentation  Hypotheses 

Accomplishing  these  objectives  entails  evaluation  of  four  hypotheses.  The  first  directly 
relates  to  Phase  1  of  the  experimentation.  The  second  and  third  will  be  evaluated  during 
the  second  phase  and  the  fourth  relates  to  the  third  phase. 

HI  -  Teams  of  unmanned  vehicles  formed  and  controlled  by  MICA  automated 
controllers  combined  with  variable  initiative  technology  does  significantly  better 
than  current  methodologies  and  tools  applied  to  unmanned  vehicles. 

This  hypothesis  relates  to  the  core  justification  for  MICA  research  and  addresses  the 
question  “Why  should  the  Government  buy  MICA?“  Further  investment  in  MICA 
research  and  experimentation  is  warranted  if  and  only  if  this  hypothesis  cannot  be  refuted 
through  experimentation.  For  this  reason,  the  first  phase  of  MICA  experimentation  is 
designed  to  evaluate  this  hypothesis  and  must  be  successfully  performed  before 
additional  experimentation  will  be  conducted. 


H2  -  MICA  controllers  provide  an  integrated  solution  spanning  each  of  the  four 
task  areas  for  a  realistic  range  of  tactical  situations. 
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The  four  task  areas  are:  1)  Team  Composition  and  Tasking  2)  Team  Dynamics  and 
Tactics,  3)  Cooperative  Path  Planning  and  4)  Uncertainty  Management.  The  range  of 
tactical  situations  shall  include  SEAD  against  Third  World  opponents  as  well  as  state-of- 
the-art  Integrated  Air  Defense  Systems,  Interdiction  against  fixed  and  moving  targets 
including  designated  Time  Critical  Targets,  Intelligence,  Surveillance  and 
Reconnaissance,  employment  of  a  variety  of  weapons,  sensors  and  countermeasures  and 
adherence  to  representative  Commander’s  Guidance  and  Rules  of  Engagement. 
Validation  against  this  hypothesis  ensures  the  robustness  and  attendant  operational  utility 
of  the  MICA  research  artifacts. 


H3  -  MICA  controllers  provide  efficient  operator  integration  enabling  safe  and 
robust  human  control  of  automa-teams  consisting  of  5  to  30  heterogeneous 
unmanned  vehicles. 

This  hypothesis  addresses  human  involvement  in  the  battlespace  management  and 
control  process.  Status  and  recommendations  provided  by  decision  support  systems  must 
be  of  interest  to  and  naturally  understandable  by  a  human,  who  must  also  be  able  to 
provide  guidance  to  automata  in  a  natural  form. 

H4  -  MICA  research  artifacts  can  be  transitioned  to  near-term  (2  to  4  years) 
operational  employment. 

The  critical  issue  regarding  validation  of  this  hypothesis  is  early  introduction  on 
hardware  experimentation.  Validation  of  the  transition  potential  of  MICA  research 
requires  that  mixes  of  heterogeneous  hardware  platforms  be  controlled  using  MICA 
algorithms  in  a  synthetic  battlespace. 

A.2.1  Hypothesis  HI 

The  goal  of  HI  experimentation  is  to  gauge  the  value  of  MICA  technology  for  controlling 
teams  of  unmanned  platforms  against  the  current  technology.  The  value  of  MICA 
technology  will  be  evaluated  against  two  hypotheses:  Hl-1)  Development  and  execution 
of  unmanned  vehicle  battle  plans  using  MICA  controllers  utilizes  significantly  fewer 
resources  than  current  tools  and  processes  to  accomplish  the  same  job;  and  Hl-2)  MICA 
controllers  provide  superior  battle  plans  and  execution  compared  to  current  tools  and 
processes.  Figure  A.  1.1-1  depicts  the  process  to  be  used  for  HI  Experimentation. 

To  evaluate  the  first  hypothesis  (Hl-1),  we  will  conduct  a  paper  experiment  at  the  133rd 
Air  National  Guard  -  Fort  Dodge  to  determine  manning  and  technology  needs.  This 
experiment  will  be  conducted  over  the  course  of  a  single  day  and  will  draw  upon  domain 
experts  at  the  133rd  to  identify  resource  requirements.  The  133rd  detachment  has 
expertise  in  air  operations  planning  for  manned  platforms.  They  will  provide  insight  into 
personnel  needs  to  accomplish  the  detailed  planning  and  coordination  tasks  that  are 
handled  autonomously  by  MICA  controllers.  Hl-1  represents  a  gate  that  will  determine 
whether  additional  HI  testing  is  required  to  quantify  the  benefit  of  MICA  technology. 
The  results  of  Hl-1  testing  will  be  an  estimate  of  people  requirements  to  perform  this 
function  including  the  logistics  train  required  to  support  this  number. 
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HI  Experimentation 


Figure  A.l.1-1  -  HI  Experimentation 

In  HI -2  experimentation,  the  set  of  domain  experts  identified  in  Hl-1  would  be  utilized 
to  plan,  coordinate,  and  execute  the  MICA  challenge  problem  using  currently  available 
tools  and  technologies.  The  same  challenge  problem  would  be  executed  using  MICA 
controller(s).  The  experimentation  for  HI -2  would  utilize  the  MICA  OEP  to  simulate  the 
dynamic  battlefield  environment  including  generation  of  representative  sensor  reports 
and  for  conflict  resolution  and  the  gathering  of  performance  metrics.  An  apples-to-apples 
comparison  between  MICA  controllers  and  current  tools  would  be  available  through 
development  of  the  same  set  of  performance  metrics.  We  currently  envision  this  testing 
to  be  performed  using  one  or  both  industrial  prime  controllers.  HI -2  testing  would  be 
performed  on  the  nominal  factors  with  a  single  iteration.  The  hypothesis  HI  indicates 
that  MICA  controllers  should  be  significantly  superior  -  thus  the  need  for  multiple 
iterations  to  generate  statistically  significant  results  is  diminished. 

HI -2  experimentation  would  require  significant  effort  from  a  set  of  domain  experts  in 
order  to  develop  and  execute  a  responsive  battle  plan.  Therefore  it  will  be  performed 
only  if  the  results  of  Hl-1  are  not  conclusive  regarding  MICA  potential  for  a  significant 
reduction  in  required  resources.  In  this  case  the  decisive  criterion  regarding  MICA 
research  value  is  the  quality  of  resultant  planning,  coordination  and  execution  relative  to 
the  current  means  identified  in  Hl-1.  To  put  this  in  simple  terms,  it  is  expected  that  Hl-1 
will  indicate  that  significantly  more  resources,  personnel,  air  vehicles,  planning  and 
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execution  time  and  logistic  support,  will  be  required  to  execute  the  challenge  problem 
using  current  tools  and  processes.  Given  this  result,  there  is  no  need  to  perform  HI -2 
testing  since  the  only  remaining  question  revolves  around  MICA  controller  performance. 
Are  MICA  controllers  capable  of  planning,  coordinating  and  executing  the  challenge 
problem  to  the  standards  defined  below  for  H2  and  H3  experimentation?  If  they 
demonstrate  this  capability,  they  are  clearly  superior  to  current  tools  and  processes  given 
that  Hl-1  shows  significant  cost  savings.  However,  if  Hl-1  indicates  that  MICA 
controllers  do  not  offer  clear  advantage  in  terms  of  resource  consumption,  it  may  be 
necessary  to  execute  HI -2  to  quantify  the  relative  performance  of  MICA  controllers 
against  the  current  tools  and  processes  and  to  compare  MICA  controllers  against  this 
standard.  In  order  to  formulate  a  clear  decision  regarding  H2-2  experimentation,  Boeing 
will  formulate  a  formal  recommendation  for  Government  review  subsequent  to 
completion  of  the  Hl-1  experiment. 


A.2.2  Hypothesis  H2 


H2  Experimentation 

Perform  for  Each  Controller 


Figure  A.  1.2-1-  H2  Experimentation 

The  intent  of  H2  experimentation  is  to  examine  the  capability  and  robustness  of  each 
MICA  controllers  across  the  full  spectrum  of  MICA  requirements  and  war  fighting 
situations.  This  is  an  enormous  challenge  involving  an  extensive  number  of 
combinations  of  experiment  factors  and  values.  There  is  obviously  a  physical  limit  to 
number  of  experiments  that  can  be  conducted  in  a  reasonable  time  frame. 
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A  typical  challenge  problem  lasts  24  -72  hours  and  the  combined  OEP  and  controller 
execute  3-5  times  real-time.  Although  extreme  performance  measure  accuracy  (with 
attendant  large  number  of  Monte  Carl  Iterations)  is  not  needed,  3  to  5  Monte  Carlo  runs 
will  be  executed  for  each  data  point.  Additionally,  there  are  a  large  numbers  of  candidate 
metrics  providing  both  technical  MOP  and  operationally  relevant  MOEs.  It  is  therefore 
critical  that  an  efficient  approach  to  H2  experimentation  be  adopted. 

Figure  A.  1.2-1  captures  the  H2  Experimentation  Process.  OEP  2.0  and  CP  2.0, 
developed  in  response  to  this  prioritization,  will  provide  the  basis  H2  experimentation. 
The  Challenge  Problem  (Version  2.0)  will  be  reformulated,  with  the  same  areas  of 
emphasis,  to  ensure  that  researchers  have  not  tuned  their  controllers  to  CP  challenges  and 
red  tactics.  In  addition,  a  series  of  experimentation  factors,  described  below,  will  be 
incorporated  through  variations  in  the  CP  models  and  scripts  and  red  controller 
capabilities  and  tactics.  Importantly,  H2  experimentation  for  each  controller  will  be 
tailored  to  the  capabilities  of  the  controller.  A  pre-screening  process  will  be  used  to 
ascertain  the  capabilities  of  each  controller  relative  to  the  experimentation  factors.  The 
experimentation  will  then  be  executed  only  to  the  extent  that  the  controller  is  capable. 
Results  of  H2  experimentation  will  be  combined  with  H3  results  and  analyzed  for 
completeness  relative  to  experimentation  factors  and  MICA  research  objectives.  An 
experimentation  report  will  be  produced  which  highlights  each  controller’s  strengths  and 
areas  for  further  research.  Theses  experimentation  results  will  be  used  to  select  the  best 
candidates  for  H4  Experimentation. 

The  H2  approach  chosen  applies  Design  of  Experiments  (DOE)  methodologies  to  reduce 
the  number  of  experiments  to  manageable  level.  The  requisite  experimentation  will  be 
executed  at  Boeing  using  the  MICA  OEP  to  exercise  each  candidate  controller  in  an 
automated  execution  of  a  limited  ensemble  of  Monte  Carlo  runs.  To  the  extent  feasible 
parallel  experimentation  will  be  executed  on  multiple  processors  to  hasten  the  process. 

The  limited  duration  Monte  Carlo  approach  is  warranted  because  we  are  interested  in 
gross  measures  of  performance  rather  than  a  precise  measure.  Boeing  has  worked  with 
the  Government  and  researchers  to  limit  the  number  of  factors  to  be  considered.  Because 
executing  even  this  limited  set  of  factors  for  each  controller  will  be  time  (and  resource) 
consuming,  a  pre-screening  process  will  be  used  to  provide  a  first  order  gate  to  determine 
completeness  of  the  controller  with  respect  to  the  MICA  challenge  problem.  Based  on 
the  results  of  this  pre-screen,  a  more  limited  set  of  factors,  tailored  to  the  specific 
capabilities  of  each  controller,  will  be  run. 

H2  testing  will  rely  upon  the  capabilities,  features,  and  war-fighting  challenges  provided 
by  the  OEP  and  Challenge  problem.  Early  in  the  MICA  Program,  Boeing  analyzed 
candidate  OEP  and  Challenge  Problem  (CP)  features  against  the  MICA  BAA 
specification  of  desired  controller  capabilities.  OEP/CP  features  considered  included: 
Variations  in  Commander’s  Guidance  and  Rule  of  Engagement,  Diversity  of  UAV, 
weapon  and  sensor  types,  Dynamic  CP  with  multiple  TCT  excursions,  Diurnal  and 
Weather  effects,  Logistic  and  Battle  Damage  Effects  and  Delays,  3-D  Air  Vehicle 
Signature,  Integrated  Air  Defense  System  (IADS)  depth  and  sophistication,  Enforcement 
of  Deconfliction  and  No-Fly  Zones,  Extent  of  white  vehicular  traffic  and  white  fixed 
assets,  Accurate  modeling  of  Blue  Weapon  Damage  to  Red  Assets,  Terrain, 
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Multilateration  of  Blue  ESM,  Blue  Cooperative  Jamming,  Accurate  model  of  sensors  and 
sensor  report  fusion  and  Reactive  Red  Ground  Controller.  Analysis  of  the  influence  of 
each  of  these  factors  on  MICA  research  objectives  yielded  four  areas  of  extreme 
importance:  1)  Dynamic  CP  with  multiple  TCT  excursions,  2)  3-D  Air  Vehicle  Signature, 
3)  Integrated  Air  Defense  System  (IADS)  depth  and  sophistication  and  4)  Accurate 
model  of  sensors  and  sensor  report  fusion.  These  four  areas  have  been  give  maximum 
emphasis  in  the  OEP/CP  development.  Seven  additional  areas  score  as  moderately 
important:  1)  Variations  in  Commander’s  Guidance  and  Rule  of  Engagement,  2) 
Diversity  of  UAV,  weapon  and  sensor  types,  3)  Logistic  and  Battle  Damage  Effects  and 
Delays,  4)  Enforcement  of  Deconfliction  and  No-Fly  Zones,  5)  Multilateration  of  Blue 
ESM,  6)  Blue  Cooperative  Jamming,  and  7)  Reactive  Red  Ground  Controller.  These 
capabilities  have  also  been  emphasized  but  less  than  the  first  four.  Finally  four  areas 
show  considerably  less  influence:  1)  Diurnal  and  Weather  effects,  2)  and  3)  Extent  of 
white  vehicular  traffic  and  white  fixed  assets,  and  4)  Accurate  modeling  of  Blue  Weapon 
Damage  to  Red  Assets.  These  have  been  given  the  least  emphasis. 


A.2.2.1  Design  of  Experiments  Methodology 

Boeing  will  tailor  the  classic  Design  of  Experiments  (DOE)  process  to  MICA  H2 
experimentation.  DOE  provides  a  methodology  designed  to  reduces  the  number  of 
experiments  required  to  characterize  complex  systems  and  processes.  This  methodology 
applies  to  the  selection  of  high  leverage  experimentation  factors  (independent  variables), 
reducing  requisite  number  of  test  points  required  for  each  selected  factor  and  analysis  of 
results  to  characterize  the  effects  of  each  factor  considered.  Appendix  A  outlines  the 
DOE  process  that  provides  the  basic  design  of  our  H2  Experimentation  approach. 


A.2.2.2  Experimentation  Factors 

In  conjunction  with  the  Government  and  other  researchers,  Boeing  has  identified  a  set 
of  candidate  factors  for  consideration  in  H2  experimentation.  These  factors  represent  the 
set  of  independent  variables  that  exert  the  greatest  influence  on  MICA  controller 
performance.  They  may  be  separated  into  five  general  categories:  1)  Variations  in  the 
balance  of  power  between  red  and  blue,  2)  Variations  in  the  quality  of  intelligence  prior 
to  initiation  of  hostilities,  3)  Variations  in  Command  and  Control  including  Commander’s 
Guidance  and  Rules  of  Engagement  (ROE),  4)  Weather  and  5)  Variations  in  rate  of 
system  and  sub-system  failures.  Within  each  of  these  categories,  multiple  candidate 
factors  were  evaluated. 


When  operational,  MICA  controllers  will  need  to  operate  in  future  battle  spaces  ranging 
from  Major  Regional  Conflicts  against  state-of-the-art  defenses  through  third-world 
interventions  with  less  sophisticated  defensive  systems.  In  all  situations  they  will  be 
required  to  control  a  heterogeneous  array  of  unmanned  vehicles  that  have  varying 
capabilities  including  their  ability  to  penetrate  enemy  air  defenses.  It  is  important, 
therefore,  that  the  controllers  be  able  to  adjust  to  these  situations  that  can  be  characterized 
as  variations  in  the  balance  of  power  between  blue  and  red.  For  this  reason,  MICA  H2 
experimentation  will  include  a  set  of  factors  that  adjust  this  balance.  Balance  of  power 
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factors  considered  for  H2  experimentation  include:  1)  Level  of  Blue  UAV  signatures,  2) 
Red  SAM  capabilities  (Search  Radar,  FC  Radar,  missile),  3)  Concentration  of  assets  in 
the  IADS,  4)  Shape  of  the  Blue  UAV  signature  envelope,  5)  Blue  jamming  capability,  6) 
Availability  and  capability  of  blue  weapons,  7)  Availability  of  decoys,  8)  Extent  armor 
and  SSM  activity  in  heavily  defended  areas,  9)  Red  air  defense  and  ground  control  tactics 
and  aggressiveness,  and  10)  Capability  of  red  air  defenses  to  detect  decoys.  After 
extensive  discussion  it  was  decided  that  varying  the  level  (not  shape)  of  the  blue  UAV 
signatures  was  the  most  effective  way  of  representing  variations  in  UAV  penetration 
capability  and  SAM  defensive  capability.  This  means  that  item  1)  will  be  included  and 
items  2),  and  4)  will  not.  Regarding  item  3),  the  IADS  composition  for  the 
experimentation  will  be  made  more  robust  than  that  used  to  date.  Item  5)  (Blue  jamming 
capability)  will  be  included  as  a  factor.  Item  6)  and  7)  will  be  included  to  the  extent  that 
researchers  will  be  given  a  fixed  (limited)  set  of  weapons  and  decoys  for  use  throughout 
the  experimentation  Challenge  Problem  scenario.  These  will  be  chosen  to  allow 
sufficient  response  to  the  planned  variations  in  Commander’s  Guidance  and  ROE. 
Similarly,  the  experimentation  Challenge  Problem  will  incorporate  items  8)  and  9)  and 
will  include  a  Red  Cell  capability  in  which  human-in-the-loop  control  can  be  exercised 
on  red  air  defense  and  ground  assets.  Item  10)  was  deemed  of  interest  and  will  be 
included  in  the  experiments  as  a  factor,  which  varies  the  red  capability  to  discern  decoys. 


Suggested  Factors 


Factor 

Expected  Benefit 

Variations 

Primary  Applicable  MICA 
Technology 

•Estimate  of  overall  Red  strength  & 
Tactics 

•Controller  Robustness  in  presence 
of  Intel  shortfalls 

•Accurate  Estimate,  50%  over 
and  50%  under  estimate 

•Uncertainty  Management 

•  Time  Varying  Guidance  and  ROE 

•Determine  responsiveness  of 
controllers  as  the  goals  and 
constraints  evolve 

•None  (SEAD,  TCT,  etc) 

•  Random  time  for  change  in 

ROE  and  CG 

•Team  Composition 
•Team  Dynamics 
•Cooperative  Path  Planning 

•Vary  time  available  to  achieve 
Commander' s  Objective 

•Inject  different  tactics  as  required 
by  Commander's  Guidance 

•Nominal 

•  12  hours  to  eliminate  C2 

Facilities 

•Team  Composition 
•Team  Dynamics 
•Cooperative  Path  Planning 

•UAV  Signature  (Variation 

Known) 

•Examine  controller  robustness  in 
MRC  vs  3rd  World  Scenario 

•10  db  above  Nominal 

•Nominal 

•Team  Dynamics 
•Cooperative  Path  Planning 

•J  ammer  Effectiveness 

•Controller  Robustness 

•Side  lobe  at  30  db  below  main 
lobe 

•Side  lobe  at  20  db  below  main 
lobe 

•Team  Dynamics 
•Cooperative  Path  Planning 

•Vary  ability  of  IADS  to  discern 
decoys 

•Controller  Robustness  in  presence 
on  realistic  battlefield  limitations 

•Nominal 

•Only  50%  of  decoys  discerned 

•Team  Composition 
•Team  Dynamics 
•Cooperative  Path  Planning 

•Random  Vehicle  Failures 

•Controller  Robustness  in  presence 
of  realistic  tactical  environment 

•No  failures 

•MTBF  of  10  hours 

•Team  Composition 
•Uncertainty  Management 

12 

Figure  A.l.2.2-1  -  Recommended  ExperimentationFactors 
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In  order  to  accommodate  operational  realities  MICA  controllers  must  be  able  to  adjust  to 
varying  levels  of  intelligence  ranging  from  almost  complete  knowledge  of  enemy 
capabilities  and  deployment  down  to  relatively  little  known.  Factors,  relating  to 
intelligence,  consider  for  inclusion  are:  1)  Percentage  of  red  assets  know  a  priori,  2) 
Accuracy  to  which  red  assets  are  known,  3)  Percentage  correct  ID  of  known  red  assets,  4) 
Accuracy  of  overall  strength  of  red  forces,  and  5)  Accuracy  of  prediction  of  red  tactics 
and  intent.  It  was  decided  that  the  first  three  items  would  be  varied  in  tandem  and  the 
latter  two  would  be  constant  for  H2  experimentation. 

Operationally  MICA  controllers  will  be  required  to  offer  closed  loop  control  of  assets  in 
response  to  guidance  provided  by  Command  and  Control  elements.  Factors  considered 
for  inclusion  in  this  area  were:  1)  Separate  static  cases  which  emphasize  SEAD  or  TCT 
suppression  (assume  manned  aircraft  and  cruise  missiles  responsible  for  SEAD  and 
UAVs  perform  suicide  missions  against  TCTs),  2)  Time  variant  Guidance  and  ROE,  3) 
Inject  real-time  intelligence  divulging  impending  SSM  attack,  and  4)  Vary  time  available 
to  achieve  objective  (example  5  days  to  eliminate  C2  Facilities  vice  6  hours).  It  was 
decided  that  H2  experimentation  would  include  time  varying  Commander’s  Guidance 
and  ROE  that  emphasizes  different  warfare  areas.  In  addition,  item  3)  will  be 
incorporated  into  the  scenario.  Item  4  will  also  be  treated  as  a  factor  with  two  levels 
evaluated  in  the  experimentation. 

Since  MICA  controllers  must  operate  in  any  weather  for  which  the  vehicles  are  capable, 
weather  was  considered  as  a  factor.  It  was  decided  that  controller  ability  to  function  in 
inclement  weather  will  be  verified  but  weather  will  not  be  included  as  a  factor  in  the  H2 
experimentation. 

The  ability  of  controllers  to  react  to  and  compensate  for  vehicle  failures  is  considered 
critical.  Vehicle  failure  rate  will  be  included  as  an  H2  experimentation  factor.  Figure 
A.  1.2. 2-1  above  lists  those  factors  chosen  for  inclusion  in  H2  experimentation  along  with 
the  variations  planned. 

A.2.2.3  Pre-Screening  Criteria 

The  following  questions  will  be  used  to  pre-screen  controllers  to  determine  the  extent  of 
experimentation  to  be  performed. 

Team  Composition  -  Discuss  degree  to  which  controller  does  the  following: 

Responds  to  static  Commander’s  Guidance  and  ROE 

•  What  is  the  mechanism  of  linking  CG  and  ROE  to  controller  performance 
and  tactics? 

Responds  to  dynamic  XML  representation  of  Commander’s  Guidance  and  ROE. 

Composes  team  from  multiple  heterogeneous  UAV  and  sensor  variants 

•  What  criteria  and  logic  are  used  to  select  team  members  and  assign  roles? 

Employs  full  spectrum  of  weapons  modeled 

•  What  criteria  and  logic  are  used  to  select  weapons  against  targets 
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•  How  is  collateral  damage  considered,  is  there  a  cost  mechanism? 

Coordinates  multi-team  composition  and  activities 

•  Are  teams  autonomous  or  do  they  collaborate? 

Team  Dynamics  &  Cooperative  Path  Planning  -  Discuss  degree  to  which  the 
controller  does  the  following: 

Incorporates  dynamic  information  into  real-time  plan  development 

Develops  survivable  penetrating  and  /  or  threat  avoidance  routes  in  three 
dimensions  against  fixed  and  mobile  threats 

•  What  cost  function  or  other  criteria  are  used 

Discovers  threat  laydowns  that  are  imprecisely  located  and  accurately  fixes 

•  How  are  paths  planned  to  ensure  proper  geometry  to  geolocate  emitting 
platforms 

Employs  jamming  and  other  countermeasures  including  decoys 

•  How  are  routes  planned  to  ensure  proper  positioning  and  timing  of 
jamming  support 

•  What  cooperative  multi -platform  jamming  techniques  used 

Employs  multiple  sensors  and  platforms  to  find,  fix,  and  identify  objects  of 
interest 

•  How  are  assets  synchronized  to  provide  multiple  looks  from  different 
geometries 

•  How  is  logic  linked  to  meet  ROE  requirements  for  identification 

Responds  to  real-time  intelligence  divulging  an  impending  TCT  attack  against 
blue  or  other  high  priority  tasking  or  re-tasking 

Varies  tactics  based  on  balance  of  power  between  red  and  blue  and  urgency  in 
achieving  tactical  objectives  conveyed  in  Commander’s  Guidance 

Uncertainty  Management  -  Discuss  degree  to  which  the  controller  does  the  following: 

Behaves  with  varying  levels  of  IPB  extent  and  quality 

•  How  is  the  IPB  discovered? 

•  Are  unique  tactics  to  stimulate  defenses  employed? 

•  How  is  this  reflected  in  team  composition  and  tactics 

Responds  to  vehicle  failures  including  communications  outages  and  battle 
damage 

•  How  are  failures  discovered 

•  How  are  failures  reflected  in  team  composition  and  tactics 
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Adapt  to  different  weather,  predicted  and  unpredicted 

Variable  Initiative 

Are  there  situations  where  an  operator  is  required  for  basic  execution  of  the 
controller  —  is  autonomous  operation  infeasible 

What  information  does  the  controller  provide  the  system  operator? 

What  decisions  and  actions  does  the  controller  require  from  the  operator? 

•  How  do  these  change  with  ROE  and  CG  updates? 

•  Is  the  amount  of  automation  adjustable  by  the  operator? 

How  does  the  HCI  monitor  battle  progress? 

•  Is  there  an  indication  on  the  degree  to  which  objectives  are  being 
achieved? 

•  Is  the  HCI  extensible  to  support  larger  and  smaller  teams? 

What  is  the  skill  set  required  for  the  operator? 

How  does  the  MICA  HCI  complement  /  conflict  with  the  vehicle  controller 

Does  the  design  envision  a  separate  controller  function  or  does  the  HCI 
incorporate  vehicle  control  functions? 

Does  the  controller  provide  all  information  required  by  your  HMI?  If  not  list 
information  that  will  be  required  from  the  OEP. 

Assumptions 

List  any  assumptions,  regarding  platforms,  sensors,  weapons  and  red  capabilities 
used  in  the  development  of  your  controller. 

A.2.2.4  Measures  of  Performance 

For  MICA  experimentation  metrics  will  focus  on  operational  values  that  show  how 
MICA  technology  improves  war  fighting  and  can  be  related  to  specific  technical 
objectives  and  over-arching  technical  performance  measures.  Examples  include:  1) 
Number  of  vehicles  per  operator,  2)  Mission  /  tasks  complete,  3)  Targets  detected, 
identified,  and  damaged  /  destroyed,  4)  Blue  platforms  damaged  /  destroyed,  5)  ROE 
violations,  collateral  damage. 

Primary  metrics  for  each  experiment  and  each  iteration  will  be  Operational  Utility  in 
terms  of  Total  System  of  System  score  (includes  value  of  targets  destroyed,  ROE 
violations,  and  damage  sustained),  2)  System  Performance  in  terms  of  Targets  destroyed 
of  each  type,  Blue  platform  losses,  ROE  violations  (No-hit”  targets  hit,  etc),  Number  and 
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type  of  red  attacks  against  blue  assets,  Collateral  damage,  and  3)  Plan  Efficiency  which 
includes  consideration  of  Time  and  number  of  sorties  to  achieve  objectives  from 
commanders  guidance  (  N  %,  100  %),  and  Blue  platform  weapons  used  by  type,  4)  How 
well  is  the  battlefield  “discovered”  by  the  controllers  which  considers  Targets  (fixed 
and  mobile)  detected,  identified,  attacked,  BDA,  Targets  not  detected  /  misidentified  /  not 
BDA’d,  Time  to  detect,  id,  attack,  and  BDA,  TCT’s  detected,  identified,  and  attacked, 
Time  to  detect,  id,  attack,  and  BDA,  Time  attack  to  BDA,  How  comprehensive  are  the 
plans  ,  %  of  target  set  covered,  Span  and  efficiency  of  controllers,  Number  of  platforms 
controlled  per  controller,  and  Amount  of  “real-time”  used  by  controller. 


•  The  following  gives  a  range  for  scoring.  The  specific  value 
the  Commander' s  Guidance: 

will  be  a  function  of 

♦ 

SSM 

50  to  100 

♦ 

SSM  Support  Facility 

200  to  300 

♦ 

Tank,  SPARTY,  Mobile  HQ,  Personnel  Carrier,  Comm  Van 

5  to  15 

♦ 

Red  Fuel  Truck 

5  to  15 

♦ 

Other  Red  Truck 

1  to  3 

♦ 

Long  SAM  node,  ESM  sensor,  or  EW  radar 

5  to  15 

♦ 

C2  Facility 

5  to  15 

♦ 

Medium  SAM 

5  to  10 

♦ 

Short  SAM  or  AAA 

4  to  8 

♦ 

EachUAV  destroyed 

-10  to  -30 

♦ 

Each  SSM  launch  against  Blue  Base 

-25  to  -75 

♦ 

Each  tank,  artillery  projectile  launch  or  small  arms  attack 

-1  to  -5 

♦ 

Each  white  asset  destroyed  or  damaged 

-10  to  -25 

♦ 

Each  ROE  violation 

-10  to  -25 

ph'  pj  o  * 

18 

Figure  A.l.2.4-1  -  Range  of  Scoring  Values 


Scores  will  be  associated  with  each  metric  and  a  composite  score  will  be  generated  for 
each  controller.  The  composite  score  will  also  be  broken  down  into  subsets  such  as 
platforms  of  each  type  detected,  identified,  damaged  and  destroyed,  ROE  violations, 
enemy  attacks  on  blue  forces,  etc.  Scoring  will  be  adjusted  to  reflect  changes  in  guidance 
and  Roe.  An  example  score  for  red  platforms  destroyed  might  be  100  points  for  SSM,  5 
for  a  tank  that  is  not  threatening  blue  assets  but  15  for  a  tank  within  attack  range  of  blue. 
Points  may  be  subtracted  for  violating  ROE  or  inflicting  collateral  damage.  Figure 
A.l.2.4-1  shows  currently  planned  scoring  values.  The  range  of  values  reflects  variations 
in  Commander’s  Guidance. 
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A.2.3  Hypothesis  H3 

H3  addresses  human  involvement  in  the  battlespace  management  and  control  process. 
Status  and  recommendations  provided  by  decision  support  systems  must  be  of  interest  to 
and  naturally  understandable  by  a  human,  who  must  also  be  able  to  provide  guidance  to 
automata  in  a  natural  form.  Most  importantly,  a  major  goal  of  MICA  research  is  to 
extend  the  operator  span  of  control  to  5  vehicles  per  operator  by  2003  and  30  vehicles  by 
2005.  Since  initial  H3  experimentation  will  occur  in  2003,  we  will  focus  on  a  span  of 
control  of  5  vehicles. 


H3  Experimentation 

Perform  for  Each  Controller 


Figure  A.l.3-1  -  H3  Experimentation 

H3  experimentation  will  be  performed  in  a  limited  set  of  experiments.  Figure  A.l.3-1 
depicts  the  process  for  H3  experimentation  that  will  be  executed  for  each  candidate 
MICA  controller.  For  all  H3  experimentation,  the  controllers  provide  all  requisite  inputs 
to  the  operator;  that  is  there  is  no  direct  interface  from  the  OEP  to  the  controller  HCI. 
However,  the  HCI  design  may  assume  that  in  a  warfighting  situation,  Essential  Elements 
of  Information  will  be  received  by  the  controller  HCI  from  operational  Intelligence 
systems  and  or  platform  sensors.  As  part  of  the  pre-screening  process,  researchers  will 
identify  any  Elements  of  Information  required  from  the  OEP  to  satisfy  requirements  of 
their  HCI.  Boeing,  in  conjunction  with  the  Government  team,  will  determine  the  extent 
to  which  these  needs  can  be  accommodated.  Because  the  focus  of  H3  experimentation 
deals  with  the  quality  of  the  human  interaction,  there  is  no  need  to  conduct  a  series  of 
Monte  Carlo  iterations.  By  its  nature  H3  experimentation  must  be  performed  in  real¬ 
time.  We  will  concentrate  on  the  subset  of  H2  experimentation  in  which  variations  in 
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Commander’s  Guidance  and  ROE  is  emphasized.  In  the  H3  testing,  one  team  of  5  air 
vehicles  will  be  selected  for  operator  control.  All  other  teams  will  operate  in  the 
automatic  control  mode  used  to  execute  H2  testing.  A  single  one-day  scenario  will  be 
executed  with  operators  transitioning  as  required.  In  addition  to  the  system  level  metrics 
and  scoring  used  for  H2  experimentation,  additional  cognitive  engineering  measures  such 
as  number  of  false  negatives,  number  of  false  positives,  and  time  delay  in  generating 
operator  decisions  will  be  collected.  The  specific  set  of  cognitive  engineering  measures, 
for  each  controller,  will  be  generated  in  collaboration  with  the  appropriate  researchers 
and  the  MICA  Variable  Initiative  Engineering  Team. 


A.2.4  Hypothesis  H4 


MICA  live  Flight  Demo  Approach 


Live  Flight  Vehicle  Data 
Encoded  on  Downlink: 

•  Vehicle  state  (e.g.,  GPS 
location) 

•  Imagery 


(Heterogeneous)  Live  Flight 
UAV(s) 

•  Capable  of  waypoint  flight 

•  on-board  EO  sensor 


Live  Flight  Vehicle  Commands 
Encoded  on  Uplink: 

•  waypoints 

•  sensor  pointing  commands 


Information  from  live 
flight  vehicles: 

•  vehicle  state 

•  target  detections 
and  locations 

- ►  LAN 


MICA  OEP 


Commands  for  Blue 
Live  Flight  Vehicles: 

*  mission  waypoints 

*  action  points 

*  sensor  pointing  and 
weapon  commands 


Battlespace  State: 

•  blue  (virtual  and 
live  flight)vvehicles 
state 

•  threats 

•  target  tracks 

LAN  - ► 


Integrates  State  of  Live/Virtual 
Battlespace  Elements: 

•  virtual  element  states 
(vehicles,  defenses,  targets) 
updated  by  OEP  models 

•  actual  vehicles  and  target 
state  inferred  from  live  data 
feeds 


Commands  for  Blue 
(Virtual  and  Live 
Flight)  Vehicles: 

•  mission  waypoints 

•  action  points 

•  sensor  and  weapon 
commands 


MICA 

Controller 

—  Ml  IB .j 

U  " 

I  s:*  -  ’ 
1$ 

*?=* —  -  3: 

Tl 

i 

! —  «- - 

Figure  A.l.4-1  -  H4  Experimentation 


H4  experimentation  is  designed  to  validate  the  transition  potential  of  MICA  research.  No 
amount  of  simulation,  regardless  of  the  fidelity,  can  totally  convince  end  users  of  MICA 
controller  capability.  Furthermore,  hardware  experimentation  is  the  essential  first  step  in 
productizing  MICA  research.  It  initiates  the  process  of  integrating  MICA  controllers  to 
hardware  platform  using  off-the-shelf  Mission  Control  Stations  as  the  interface.  Only 
through  hardware  experimentation  can  questions  be  completely  resolved  regarding  the 
interrelationship  between  real  world  effects  and  MICA  controller  performance.  These 
real  world  effects  include:  1)  Inexactness  in  vehicle  trajectory  control  due  to  real  world 
navigation  inaccuracies,  wind  and  gusts  enroute;  2)  Effects  of  terrain,  landscape, 
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vegetation  and  cultural  features,  3)  Discriminating  real  world  fixed  and  moving  white 
objects  from  candidate  targets  (it  is  impossible  to  simulate  the  preponderance  of  white 
objects);  4)  Representative  vehicle  system  and  sub-system  failures,  5)  Realistic 
communications  delays,  bottlenecks  and  obstructions  and  6)  Realism  in  converting 
MICA  controller  outputs  into  executable  vehicle  commands. 

Executing  meaningful  H4  experimentation  requires  that  mixes  of  heterogeneous 
hardware  platforms  be  controlled  using  MICA  algorithms  in  a  synthetic  battlespace.  H4 
experimentation  will  be  performed  in  a  series  of  experiments  of  increasing  complexity 
and  fidelity.  As  the  experimentation  progresses,  the  number  and  varieties  of  included 
hardware  platforms  will  increase,  the  time  duration  of  the  experiments  will  increase,  the 
complexity  of  the  scenario  and  of  the  tasking  assigned  to  hardware  platforms  will 
increase  and  MICA  operators  (variable  initiative)  will  be  included  for  some  experiments. 
The  specific  platforms,  scenarios  and  experiments  will  be  identified  as  the 
experimentation  progresses.  However,  regardless  of  these  experimentation  factors,  the 
basic  approach  will  remain  constant  and  is  shown  in  Figure  A.  1.4-1. 

Key  elements  of  the  H4  experimentation  approach  are:  1)  the  OEP  simulates  the  entire 
battlespace  other  than  the  dynamics  of  the  hardware  platforms,  2)  The  OEP  integrates 
hardware  platform  state  into  the  synthetic  battlespace  and  generates  appropriate  synthetic 
sensor  measurements  on  the  hardware  platform  (egg  synthetic  SAM  radar  tracking 
hardware  platform),  3)  Controllers  determine  all  air  vehicle  activities  (synthetic  and  real) 
and  communicate  vehicle  commands  to  the  OEP  through  the  same  API  used  for  H2  and 
H3  experimentation,  4)  OEP  controls  synthetic  platforms  IAW  controller  commands,  5) 
The  OEP  interfaces  to  specific  hardware  platform  Mission  Control  Stations  to  command 
vehicle  activities  and  receive  sensor  outputs,  6)  For  variable  initiative  testing  controllers 
will  provide  all  information  for  their  operator  interface  including  display  of  hardware 
platform  sensor  outputs,  and  7)  As  required  operators  will  interface  with  Mission  Control 
Platforms  to  extract  information  from  sensor  outputs.  The  following  table  addresses  how 


each  OEP  function  will  1 

ie  modeled  for  hardware  platforms. 

Function 

Approach  for  Hardware  Platform  Experimentation 

Synthetic  Battlespace 

Flight  range  for  hardware  platforms  will  be  embedded  as  area 
within  Challenge  Problem  gaming  area.  Objects  of  interest 
within  range  (buildings,  vehicles,  SAMs,  etc.)  will  be 
represented  in  synthetic  space.  Additional  objects  that  are 
included  in  the  Challenge  Problem  but  not  available,  as  physical 
entities  within  the  range  will  also  be  modeled  in  the  simulated 
space.  Range  terrain  data  will  be  seamlessly  integrated  into  the 
synthetic  model.  Hardware  platforms  will  be  modeled  at  their 
true  (reported  GPS)  positions. 

Blue  engagement  by 
Red  SAM 

OEP  uses  true  position,  velocity  and  attitude  of  hardware 
platform  in  conjunction  with  simulated  3-D  signature,  SAM 
location,  and  SAM  capabilities.  SAM  tactics  will  be  controlled 
by  Red  Cell  using  automated  IADS.  Human  Red  command 
may  be  included. 
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B  Appendix  B  -  Design  of  Experiments  Methodology 


MICA  DOE  Process  Flow 


1.  Objective  definition 

a.  Evaluation:  Stated  in  MICA  Hypotheses 

b.  Comparative  designs 

i.  Choose  between  alternatives,  with  narrow  scope,  suitable  for 
an  initial  comparison 

ii. Choose  between  alternatives,  with  broad  scope,  suitable  for  a 
confirmatory  comparison 

iii. If  you  have  one  or  several  factors  under  investigation,  but  the 
primary  goal  of  your  experiment  is  to  make  a  conclusion 
about  one  a-priori  important  factor 

c.  Screening  designs  to  identify  which  factors/effects  are  important 

i. When  you  have  2-4  factors  and  can  perform  a  full  factorial 

ii. When  you  have  more  than  3  factors  and  want  to  begin  with 
as  small  a  design  as  possible 

iii. When  you  have  some  qualitative  factors,  or  you  have  some 
quantitative  factors  that  are  known  to  have  a  non-monotonic 
effect. 

d.  Response  Surface  modeling  to  achieve  one  or  more  of  the 
following 

i. Hit  a  target  objective 

ii.  Maximize  or  minimize  a  response 

iii. Reduce  variation  by  locating  a  region  where  the  process  is 
easier  to  manage. 

iv. Robustness 

e.  Regression  modeling 

i.To  estimate  a  precise  model,  quantifying  the  dependence  of 
response  variable(s)  on  process  inputs. 

2.  Selecting  Process  variables 

a.  Process  variables  include  both  inputs  and  outputs  -  i.e.,  factors  and 
responses. 

b.  Include  all  important  factors  (based  on  MICA  experts) 

c.  Check  the  factor  settings  for  impractical  or  impossible 
combinations. 

d.  Include  all  relevant  responses. 

e.  Avoid  using  only  responses  that  combine  two  or  more 
measurements  of  the  process. 

3.  Selecting  design  approach 

a.  Pre-screen  process  utilizes  fractional  factorial  design  matrices 

i.Mixed  level  Fractional  design  matrices  support  MICA 
hypotheses  which  maximum  observable  information  through 
a  minimum  number  of  runs. 
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1.  2A(k-p) 

2.  Plackett-Burman 

*-  ii. Calculate  factor  effects  and  identify  insignificant  factors  and 

factor  interactions 

iii.Refine  design  matrix  (full/fractional  orthogonal  design 
matrix) 

—  iv.Perform  Fold-Over  if  necessary. 
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Figure  B-l  -  Two-Level,  Three-Factor  Diagram 

b.  Full  factorial  design  to  evaluate  main  effects 

i. Two/Three  level  designs  requiring  2,3Afactors  experiments 

ii. Replication:  For  each  comer  of  the  box  (ran),  calculate  the 
average  response  and  observe  dispersion. 

iii. Determine  if  variance  is  homogeneous  (uniform)  across  the 
factor  space. 

iv.  Two  level,  three  factor  diagram  shown  in  Figure  B-l. 


4.  Data  Analysis 

Figure  B-2  shows  the  flow  of  the  data  analysis  process 
a.  Analysis  Steps 

i.  Graphical  construction 

1.  Histogram:  Graphically  summarize  the  distribution 
of  a  univariate  data  set. 

2.  Box:  Conveying  location  and  variation  information 
in  data  sets,  particularly  for  detecting  and  illustrating 
location  and  variation  changes  between  different  groups 
of  data. 
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3.  Dex  Scatter:  What  are  the  most  important  factors  - 
What  is  the  best  setting  for  each  of  these  important 
factors  -  What  data  points  are  outliers. 

ii.  Create  a  model  from  the  data 

1.  y  =  y  +  (Ahigh  -  Alow )  +  (Blugh  - Blow)  + ... 

iii.  Use  results  to  answer  the  questions  in  MICA  objectives 
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Figure  B-2  -  DOE  Flow 
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